Predict Bike Sharing Demand with AutoGluon Template¶
Project: Predict Bike Sharing Demand with AutoGluon¶
This notebook is a template with each step that you need to complete for the project.
Please fill in your code where there are explicit ? markers in the notebook. You are welcome to add more cells and code as you see fit.
Once you have completed all the code implementations, please export your notebook as a HTML file so the reviews can view your code. Make sure you have all outputs correctly outputted.
File-> Export Notebook As... -> Export Notebook as HTML
There is a writeup to complete as well after all code implememtation is done. Please answer all questions and attach the necessary tables and charts. You can complete the writeup in either markdown or PDF.
Completing the code template and writeup template will cover all of the rubric points for this project.
The rubric contains "Stand Out Suggestions" for enhancing the project beyond the minimum requirements. The stand out suggestions are optional. If you decide to pursue the "stand out suggestions", you can include the code in this notebook and also discuss the results in the writeup file.
Step 1: Create an account with Kaggle¶
Create Kaggle Account and download API key¶
Below is example of steps to get the API username and key. Each student will have their own username and key.
- Open account settings.
- Scroll down to API and click Create New API Token.
- Open up
kaggle.jsonand use the username and key.
Step 2: Download the Kaggle dataset using the kaggle python library¶
Open up Sagemaker Studio and use starter template¶
- Notebook should be using a
ml.t3.mediuminstance (2 vCPU + 4 GiB) - Notebook should be using kernal:
Python 3 (MXNet 1.8 Python 3.7 CPU Optimized)
Install packages¶
In [1]:
!pip install -U pip
!pip install -U setuptools wheel
!pip install -U "mxnet<2.0.0" bokeh==2.0.1
!pip install autogluon --no-cache-dir
# Without --no-cache-dir, smaller aws instances may have trouble installing
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (22.1) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (59.5.0) Collecting setuptools Using cached setuptools-62.3.2-py3-none-any.whl (1.2 MB) Requirement already satisfied: wheel in /usr/local/lib/python3.7/site-packages (0.37.1) Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 59.5.0 Uninstalling setuptools-59.5.0: Successfully uninstalled setuptools-59.5.0 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. autogluon-text 0.4.1 requires setuptools<=59.5.0, but you have setuptools 62.3.2 which is incompatible. Successfully installed setuptools-62.3.2 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Requirement already satisfied: mxnet<2.0.0 in /usr/local/lib/python3.7/site-packages (1.9.1) Requirement already satisfied: bokeh==2.0.1 in /usr/local/lib/python3.7/site-packages (2.0.1) Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (1.21.6) Requirement already satisfied: packaging>=16.8 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (21.3) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (2.8.2) Requirement already satisfied: PyYAML>=3.10 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (5.4.1) Requirement already satisfied: tornado>=5 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (6.1) Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (4.0.1) Requirement already satisfied: Jinja2>=2.7 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (3.0.3) Requirement already satisfied: pillow>=4.0 in /usr/local/lib/python3.7/site-packages (from bokeh==2.0.1) (9.0.1) Requirement already satisfied: graphviz<0.9.0,>=0.8.1 in /usr/local/lib/python3.7/site-packages (from mxnet<2.0.0) (0.8.4) Requirement already satisfied: requests<3,>=2.20.0 in /usr/local/lib/python3.7/site-packages (from mxnet<2.0.0) (2.22.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from Jinja2>=2.7->bokeh==2.0.1) (2.0.1) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/site-packages (from packaging>=16.8->bokeh==2.0.1) (3.0.6) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil>=2.1->bokeh==2.0.1) (1.16.0) Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (2.8) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (1.25.11) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests<3,>=2.20.0->mxnet<2.0.0) (2021.10.8) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Requirement already satisfied: autogluon in /usr/local/lib/python3.7/site-packages (0.4.1) Requirement already satisfied: autogluon.text==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1) Requirement already satisfied: autogluon.vision==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1) Requirement already satisfied: autogluon.core[all]==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1) Requirement already satisfied: autogluon.features==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1) Requirement already satisfied: autogluon.tabular[all]==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon) (0.4.1) Requirement already satisfied: autogluon.common==0.4.1 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (0.4.1) Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (2.22.0) Requirement already satisfied: scipy<1.8.0,>=1.5.4 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.7.3) Requirement already satisfied: numpy<1.23,>=1.21 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.21.6) Requirement already satisfied: pandas<1.4,>=1.2.5 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.3.4) Requirement already satisfied: dask<=2021.11.2,>=2021.09.1 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (2021.11.2) Requirement already satisfied: tqdm>=4.38.0 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (4.64.0) Requirement already satisfied: scikit-learn<1.1,>=1.0.0 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.0.1) Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.20.17) Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (3.5.0) Requirement already satisfied: distributed<=2021.11.2,>=2021.09.1 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (2021.11.2) Requirement already satisfied: ray<1.11,>=1.10 in /usr/local/lib/python3.7/site-packages (from autogluon.core[all]==0.4.1->autogluon) (1.10.0) Requirement already satisfied: psutil<5.9,>=5.7.3 in /usr/local/lib/python3.7/site-packages (from autogluon.features==0.4.1->autogluon) (5.8.0) Requirement already satisfied: networkx<3.0,>=2.3 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (2.6.3) Requirement already satisfied: torch<1.11,>=1.0 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (1.10.2) Requirement already satisfied: catboost<1.1,>=1.0 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (1.0.6) Requirement already satisfied: fastai<2.6,>=2.3.1 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (2.5.6) Requirement already satisfied: xgboost<1.5,>=1.4 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (1.4.2) Requirement already satisfied: lightgbm<3.4,>=3.3 in /usr/local/lib/python3.7/site-packages (from autogluon.tabular[all]==0.4.1->autogluon) (3.3.2) Requirement already satisfied: autogluon-contrib-nlp==0.0.1b20220208 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.0.1b20220208) Requirement already satisfied: scikit-image<0.20.0,>=0.19.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.19.2) Requirement already satisfied: nptyping<1.5.0,>=1.4.4 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (1.4.4) Requirement already satisfied: smart-open<5.3.0,>=5.2.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (5.2.1) Requirement already satisfied: torchmetrics<0.8.0,>=0.7.2 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.7.3) Requirement already satisfied: pytorch-lightning<1.7.0,>=1.5.10 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (1.6.3) Collecting setuptools<=59.5.0 Downloading setuptools-59.5.0-py3-none-any.whl (952 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 952.4/952.4 kB 162.3 MB/s eta 0:00:00 Requirement already satisfied: omegaconf<2.2.0,>=2.1.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (2.1.2) Requirement already satisfied: timm<0.6.0 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.5.4) Requirement already satisfied: transformers<4.17.0,>=4.16.2 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (4.16.2) Requirement already satisfied: fairscale<0.5.0,>=0.4.5 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.4.6) Requirement already satisfied: Pillow<9.1.0,>=9.0.1 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (9.0.1) Requirement already satisfied: sentencepiece<0.2.0,>=0.1.95 in /usr/local/lib/python3.7/site-packages (from autogluon.text==0.4.1->autogluon) (0.1.95) Requirement already satisfied: gluoncv<0.10.6,>=0.10.5 in /usr/local/lib/python3.7/site-packages (from autogluon.vision==0.4.1->autogluon) (0.10.5.post0) Requirement already satisfied: yacs>=0.1.6 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.1.8) Requirement already satisfied: protobuf in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (3.19.1) Requirement already satisfied: regex in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2022.4.24) Requirement already satisfied: flake8 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (4.0.1) Requirement already satisfied: sacremoses>=0.0.38 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.0.53) Requirement already satisfied: pyarrow in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (6.0.1) Requirement already satisfied: sacrebleu in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.1.0) Requirement already satisfied: tokenizers>=0.9.4 in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.12.1) Requirement already satisfied: contextvars in /usr/local/lib/python3.7/site-packages (from autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.4) Requirement already satisfied: six in /usr/local/lib/python3.7/site-packages (from catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (1.16.0) Requirement already satisfied: plotly in /usr/local/lib/python3.7/site-packages (from catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (5.4.0) Requirement already satisfied: graphviz in /usr/local/lib/python3.7/site-packages (from catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (0.8.4) Requirement already satisfied: partd>=0.3.10 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.2.0) Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.0.0) Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (5.4.1) Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2021.11.1) Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (0.11.2) Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/site-packages (from dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (21.3) Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.0.3) Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (3.0.3) Requirement already satisfied: tornado>=5 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (6.1) Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.4.0) Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (8.1.3) Requirement already satisfied: tblib>=1.6.0 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.7.0) Requirement already satisfied: zict>=0.1.3 in /usr/local/lib/python3.7/site-packages (from distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.2.0) Requirement already satisfied: fastcore<1.5,>=1.3.27 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.4.3) Requirement already satisfied: torchvision>=0.8.2 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.11.3) Requirement already satisfied: spacy<4 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (2.3.7) Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (22.1) Requirement already satisfied: fastprogress>=0.2.4 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.2) Requirement already satisfied: fastdownload<2,>=0.0.5 in /usr/local/lib/python3.7/site-packages (from fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.0.6) Requirement already satisfied: autocfg in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.4.1->autogluon) (0.0.8) Requirement already satisfied: portalocker in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.4.1->autogluon) (2.3.2) Requirement already satisfied: opencv-python in /usr/local/lib/python3.7/site-packages (from gluoncv<0.10.6,>=0.10.5->autogluon.vision==0.4.1->autogluon) (4.5.4.60) Requirement already satisfied: wheel in /usr/local/lib/python3.7/site-packages (from lightgbm<3.4,>=3.3->autogluon.tabular[all]==0.4.1->autogluon) (0.37.1) Requirement already satisfied: typish>=1.7.0 in /usr/local/lib/python3.7/site-packages (from nptyping<1.5.0,>=1.4.4->autogluon.text==0.4.1->autogluon) (1.9.3) Requirement already satisfied: antlr4-python3-runtime==4.8 in /usr/local/lib/python3.7/site-packages (from omegaconf<2.2.0,>=2.1.1->autogluon.text==0.4.1->autogluon) (4.8) Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/site-packages (from pandas<1.4,>=1.2.5->autogluon.core[all]==0.4.1->autogluon) (2.8.2) Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/site-packages (from pandas<1.4,>=1.2.5->autogluon.core[all]==0.4.1->autogluon) (2021.3) Requirement already satisfied: typing-extensions>=4.0.0 in /usr/local/lib/python3.7/site-packages (from pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (4.0.1) Requirement already satisfied: pyDeprecate<0.4.0,>=0.3.1 in /usr/local/lib/python3.7/site-packages (from pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.3.2) Requirement already satisfied: tensorboard>=2.2.0 in /usr/local/lib/python3.7/site-packages (from pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (2.9.0) Requirement already satisfied: redis>=3.5.0 in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (4.3.1) Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (4.5.1) Requirement already satisfied: filelock in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (3.7.0) Requirement already satisfied: grpcio>=1.28.1 in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (1.46.3) Requirement already satisfied: attrs in /usr/local/lib/python3.7/site-packages (from ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (21.2.0) Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.text==0.4.1->autogluon) (2021.11.2) Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.text==0.4.1->autogluon) (1.3.0) Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.7/site-packages (from scikit-image<0.20.0,>=0.19.1->autogluon.text==0.4.1->autogluon) (2.13.1) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/site-packages (from scikit-learn<1.1,>=1.0.0->autogluon.core[all]==0.4.1->autogluon) (3.0.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/site-packages (from scikit-learn<1.1,>=1.0.0->autogluon.core[all]==0.4.1->autogluon) (1.1.0) Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/site-packages (from transformers<4.17.0,>=4.16.2->autogluon.text==0.4.1->autogluon) (4.2.0) Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /usr/local/lib/python3.7/site-packages (from transformers<4.17.0,>=4.16.2->autogluon.text==0.4.1->autogluon) (0.6.0) Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.4.1->autogluon) (0.5.0) Requirement already satisfied: botocore<1.24.0,>=1.23.17 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.4.1->autogluon) (1.23.17) Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3->autogluon.core[all]==0.4.1->autogluon) (0.10.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (1.3.2) Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (3.0.6) Requirement already satisfied: setuptools-scm>=4 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (6.3.2) Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (4.28.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/site-packages (from matplotlib->autogluon.core[all]==0.4.1->autogluon) (0.11.0) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (2021.10.8) Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (2.8) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/site-packages (from requests->autogluon.core[all]==0.4.1->autogluon) (1.25.11) Requirement already satisfied: aiohttp in /usr/local/lib/python3.7/site-packages (from fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (3.8.1) Requirement already satisfied: locket in /usr/local/lib/python3.7/site-packages (from partd>=0.3.10->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.0.0) Requirement already satisfied: async-timeout>=4.0.2 in /usr/local/lib/python3.7/site-packages (from redis>=3.5.0->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (4.0.2) Requirement already satisfied: deprecated>=1.2.3 in /usr/local/lib/python3.7/site-packages (from redis>=3.5.0->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (1.2.13) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/site-packages (from importlib-metadata->transformers<4.17.0,>=4.16.2->autogluon.text==0.4.1->autogluon) (3.6.0) Requirement already satisfied: tomli>=1.0.0 in /usr/local/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib->autogluon.core[all]==0.4.1->autogluon) (1.2.2) Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (3.0.6) Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.1.3) Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.7) Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.7.7) Requirement already satisfied: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.5) Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (2.0.6) Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (0.9.1) Requirement already satisfied: thinc<7.5.0,>=7.4.1 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (7.4.5) Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.7/site-packages (from spacy<4->fastai<2.6,>=2.3.1->autogluon.tabular[all]==0.4.1->autogluon) (1.0.0) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.4.6) Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (1.0.0) Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (2.6.6) Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (2.0.2) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (3.3.4) Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.6.1) Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/site-packages (from tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (1.8.1) Requirement already satisfied: heapdict in /usr/local/lib/python3.7/site-packages (from zict>=0.1.3->distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.0.1) Requirement already satisfied: immutables>=0.9 in /usr/local/lib/python3.7/site-packages (from contextvars->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.17) Requirement already satisfied: pyflakes<2.5.0,>=2.4.0 in /usr/local/lib/python3.7/site-packages (from flake8->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.4.0) Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/site-packages (from flake8->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.6.1) Requirement already satisfied: pycodestyle<2.9.0,>=2.8.0 in /usr/local/lib/python3.7/site-packages (from flake8->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (2.8.0) Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/site-packages (from jinja2->distributed<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.0.1) Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/site-packages (from jsonschema->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (0.18.1) Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/site-packages (from jsonschema->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (5.7.1) Requirement already satisfied: tenacity>=6.2.0 in /usr/local/lib/python3.7/site-packages (from plotly->catboost<1.1,>=1.0->autogluon.tabular[all]==0.4.1->autogluon) (8.0.1) Requirement already satisfied: colorama in /usr/local/lib/python3.7/site-packages (from sacrebleu->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.4.3) Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.7/site-packages (from sacrebleu->autogluon-contrib-nlp==0.0.1b20220208->autogluon.text==0.4.1->autogluon) (0.8.9) Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.7/site-packages (from deprecated>=1.2.3->redis>=3.5.0->ray<1.11,>=1.10->autogluon.core[all]==0.4.1->autogluon) (1.14.1) Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (4.7.2) Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (5.1.0) Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.2.8) Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (1.3.1) Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (0.13.0) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (6.0.2) Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.7.2) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.3.0) Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (2.0.12) Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.7/site-packages (from aiohttp->fsspec>=0.6.0->dask<=2021.11.2,>=2021.09.1->autogluon.core[all]==0.4.1->autogluon) (1.2.0) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (0.4.8) Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning<1.7.0,>=1.5.10->autogluon.text==0.4.1->autogluon) (3.2.0) Installing collected packages: setuptools Attempting uninstall: setuptools Found existing installation: setuptools 62.3.2 Uninstalling setuptools-62.3.2: Successfully uninstalled setuptools-62.3.2 Successfully installed setuptools-59.5.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Setup Kaggle API Key¶
In [2]:
# create the .kaggle directory and an empty kaggle.json file
!mkdir -p /root/.kaggle
!touch /root/.kaggle/kaggle.json
!chmod 600 /root/.kaggle/kaggle.json
In [26]:
# Fill in your user name and key from creating the kaggle account and API token file
!pip install python-dotenv
import json
from dotenv import dotenv_values
CONFIG = dotenv_values('.env')
kaggle_username = CONFIG['KAGGLE_USERNAME']
kaggle_key = CONFIG['KAGGLE_KEY']
# Save API token the kaggle.json file
with open("/root/.kaggle/kaggle.json", "w") as f:
f.write(json.dumps({"username": kaggle_username, "key": kaggle_key}))
Requirement already satisfied: python-dotenv in /usr/local/lib/python3.7/site-packages (0.20.0) WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Download and explore dataset¶
Go to the bike sharing demand competition and agree to the terms¶
In [4]:
# Download the dataset, it will be in a .zip file so you'll need to unzip it as well.
!pip install kaggle
!kaggle competitions download -c bike-sharing-demand
# If you already downloaded it you can use the -o command to overwrite the file
!unzip -o bike-sharing-demand.zip
Collecting kaggle Using cached kaggle-1.5.12-py3-none-any.whl Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.25.11) Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/site-packages (from kaggle) (1.16.0) Requirement already satisfied: tqdm in /usr/local/lib/python3.7/site-packages (from kaggle) (4.64.0) Requirement already satisfied: certifi in /usr/local/lib/python3.7/site-packages (from kaggle) (2021.10.8) Collecting python-slugify Using cached python_slugify-6.1.2-py2.py3-none-any.whl (9.4 kB) Requirement already satisfied: requests in /usr/local/lib/python3.7/site-packages (from kaggle) (2.22.0) Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/site-packages (from kaggle) (2.8.2) Collecting text-unidecode>=1.3 Using cached text_unidecode-1.3-py2.py3-none-any.whl (78 kB) Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (2.8) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/site-packages (from requests->kaggle) (3.0.4) Installing collected packages: text-unidecode, python-slugify, kaggle Successfully installed kaggle-1.5.12 python-slugify-6.1.2 text-unidecode-1.3 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv 401 - Unauthorized Archive: bike-sharing-demand.zip inflating: sampleSubmission.csv inflating: test.csv inflating: train.csv
In [6]:
# import libriries
import pandas as pd
import tqdm as notebook_tqdm
from autogluon.tabular import TabularPredictor
In [7]:
# Create the train dataset in pandas by reading the csv
# Set the parsing of the datetime column so you can use some of the `dt` features in pandas later
train = pd.read_csv('./train.csv', parse_dates=['datetime'])
train.head()
Out[7]:
| datetime | season | holiday | workingday | weather | temp | atemp | humidity | windspeed | casual | registered | count | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2011-01-01 00:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 81 | 0.0 | 3 | 13 | 16 |
| 1 | 2011-01-01 01:00:00 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 8 | 32 | 40 |
| 2 | 2011-01-01 02:00:00 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 5 | 27 | 32 |
| 3 | 2011-01-01 03:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 3 | 10 | 13 |
| 4 | 2011-01-01 04:00:00 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 0 | 1 | 1 |
In [8]:
# Simple output of the train dataset to view some of the min/max/varition of the dataset features.
train.describe()
Out[8]:
| season | holiday | workingday | weather | temp | atemp | humidity | windspeed | casual | registered | count | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 10886.000000 | 10886.000000 | 10886.000000 | 10886.000000 | 10886.00000 | 10886.000000 | 10886.000000 | 10886.000000 | 10886.000000 | 10886.000000 | 10886.000000 |
| mean | 2.506614 | 0.028569 | 0.680875 | 1.418427 | 20.23086 | 23.655084 | 61.886460 | 12.799395 | 36.021955 | 155.552177 | 191.574132 |
| std | 1.116174 | 0.166599 | 0.466159 | 0.633839 | 7.79159 | 8.474601 | 19.245033 | 8.164537 | 49.960477 | 151.039033 | 181.144454 |
| min | 1.000000 | 0.000000 | 0.000000 | 1.000000 | 0.82000 | 0.760000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 1.000000 |
| 25% | 2.000000 | 0.000000 | 0.000000 | 1.000000 | 13.94000 | 16.665000 | 47.000000 | 7.001500 | 4.000000 | 36.000000 | 42.000000 |
| 50% | 3.000000 | 0.000000 | 1.000000 | 1.000000 | 20.50000 | 24.240000 | 62.000000 | 12.998000 | 17.000000 | 118.000000 | 145.000000 |
| 75% | 4.000000 | 0.000000 | 1.000000 | 2.000000 | 26.24000 | 31.060000 | 77.000000 | 16.997900 | 49.000000 | 222.000000 | 284.000000 |
| max | 4.000000 | 1.000000 | 1.000000 | 4.000000 | 41.00000 | 45.455000 | 100.000000 | 56.996900 | 367.000000 | 886.000000 | 977.000000 |
In [9]:
# Create the test pandas dataframe in pandas by reading the csv, remember to parse the datetime!
test = pd.read_csv('./test.csv', parse_dates=['datetime'])
test.head()
Out[9]:
| datetime | season | holiday | workingday | weather | temp | atemp | humidity | windspeed | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 2011-01-20 00:00:00 | 1 | 0 | 1 | 1 | 10.66 | 11.365 | 56 | 26.0027 |
| 1 | 2011-01-20 01:00:00 | 1 | 0 | 1 | 1 | 10.66 | 13.635 | 56 | 0.0000 |
| 2 | 2011-01-20 02:00:00 | 1 | 0 | 1 | 1 | 10.66 | 13.635 | 56 | 0.0000 |
| 3 | 2011-01-20 03:00:00 | 1 | 0 | 1 | 1 | 10.66 | 12.880 | 56 | 11.0014 |
| 4 | 2011-01-20 04:00:00 | 1 | 0 | 1 | 1 | 10.66 | 12.880 | 56 | 11.0014 |
In [10]:
# Same thing as train and test dataset
submission = pd.read_csv('./sampleSubmission.csv')
submission.head()
Out[10]:
| datetime | count | |
|---|---|---|
| 0 | 2011-01-20 00:00:00 | 0 |
| 1 | 2011-01-20 01:00:00 | 0 |
| 2 | 2011-01-20 02:00:00 | 0 |
| 3 | 2011-01-20 03:00:00 | 0 |
| 4 | 2011-01-20 04:00:00 | 0 |
In [11]:
# check the columns used for training
train.columns
Out[11]:
Index(['datetime', 'season', 'holiday', 'workingday', 'weather', 'temp',
'atemp', 'humidity', 'windspeed', 'casual', 'registered', 'count'],
dtype='object')
In [12]:
# check the columns used for testing
test.columns
Out[12]:
Index(['datetime', 'season', 'holiday', 'workingday', 'weather', 'temp',
'atemp', 'humidity', 'windspeed'],
dtype='object')
In [13]:
# check the size of each dataset
train.shape, test.shape
Out[13]:
((10886, 12), (6493, 9))
Step 3: Train a model using AutoGluon’s Tabular Prediction¶
Requirements:
- We are predicting
count, so it is the label we are setting. - Ignore
casualandregisteredcolumns as they are also not present in the test dataset. - Use the
root_mean_squared_erroras the metric to use for evaluation. - Set a time limit of 10 minutes (600 seconds).
- Use the preset
best_qualityto focus on creating the best model.
In [14]:
predictor = TabularPredictor(
label='count',
problem_type='regression',
eval_metric='root_mean_squared_error',
learner_kwargs={"ignored_columns": ["casual", "registered"]},
).fit(
train_data=train,
time_limit=600,
presets='best_quality'
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_115203/"
Presets specified: ['best_quality']
Beginning AutoGluon training ... Time limit = 600s
AutoGluon will save models to "AutogluonModels/ag-20220521_115203/"
AutoGluon Version: 0.4.1
Python Version: 3.7.10
Operating System: Linux
Train Data Rows: 10886
Train Data Columns: 11
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 2705.09 MB
Train Data (Original) Memory Usage: 0.78 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 2 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting DatetimeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('datetime', []) : 1 | ['datetime']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 5 | ['season', 'holiday', 'workingday', 'weather', 'humidity']
Types of features in processed data (raw dtype, special dtypes):
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 3 | ['season', 'weather', 'humidity']
('int', ['bool']) : 2 | ['holiday', 'workingday']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
0.4s = Fit runtime
9 features in original data used to generate 13 features in processed data.
Train Data (Processed) Memory Usage: 0.98 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.52s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 11 L1 models ...
Fitting model: KNeighborsUnif_BAG_L1 ... Training model for up to 399.55s of the 599.48s of remaining time.
-101.5462 = Validation score (root_mean_squared_error)
0.03s = Training runtime
0.1s = Validation runtime
Fitting model: KNeighborsDist_BAG_L1 ... Training model for up to 399.17s of the 599.1s of remaining time.
-84.1251 = Validation score (root_mean_squared_error)
0.03s = Training runtime
0.1s = Validation runtime
Fitting model: LightGBMXT_BAG_L1 ... Training model for up to 398.8s of the 598.72s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
2022-05-21 11:52:10,264 WARNING services.py:1866 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=0.87gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.
-131.4609 = Validation score (root_mean_squared_error)
63.86s = Training runtime
6.67s = Validation runtime
Fitting model: LightGBM_BAG_L1 ... Training model for up to 323.38s of the 523.31s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-131.0542 = Validation score (root_mean_squared_error)
27.5s = Training runtime
1.28s = Validation runtime
Fitting model: RandomForestMSE_BAG_L1 ... Training model for up to 292.37s of the 492.29s of remaining time.
-116.5443 = Validation score (root_mean_squared_error)
10.81s = Training runtime
0.85s = Validation runtime
Fitting model: CatBoost_BAG_L1 ... Training model for up to 277.91s of the 477.84s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-130.5106 = Validation score (root_mean_squared_error)
201.26s = Training runtime
0.17s = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L1 ... Training model for up to 73.22s of the 273.15s of remaining time.
-124.5881 = Validation score (root_mean_squared_error)
4.85s = Training runtime
0.52s = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L1 ... Training model for up to 65.21s of the 265.14s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-138.0336 = Validation score (root_mean_squared_error)
66.16s = Training runtime
0.45s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 196.13s of remaining time.
-84.1251 = Validation score (root_mean_squared_error)
0.64s = Training runtime
0.0s = Validation runtime
Fitting 9 L2 models ...
Fitting model: LightGBMXT_BAG_L2 ... Training model for up to 195.4s of the 195.38s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-60.1464 = Validation score (root_mean_squared_error)
53.09s = Training runtime
3.42s = Validation runtime
Fitting model: LightGBM_BAG_L2 ... Training model for up to 138.3s of the 138.28s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-55.0493 = Validation score (root_mean_squared_error)
26.54s = Training runtime
0.24s = Validation runtime
Fitting model: RandomForestMSE_BAG_L2 ... Training model for up to 108.93s of the 108.91s of remaining time.
-53.4369 = Validation score (root_mean_squared_error)
26.17s = Training runtime
0.59s = Validation runtime
Fitting model: CatBoost_BAG_L2 ... Training model for up to 79.64s of the 79.62s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-55.7246 = Validation score (root_mean_squared_error)
70.12s = Training runtime
0.08s = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L2 ... Training model for up to 6.92s of the 6.9s of remaining time.
-53.841 = Validation score (root_mean_squared_error)
8.21s = Training runtime
0.59s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the -4.61s of remaining time.
-52.8219 = Validation score (root_mean_squared_error)
0.45s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 605.29s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_115203/")
Review AutoGluon's training run with ranking of models that did the best.¶
In [15]:
predictor.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -52.821924 11.568244 435.880349 0.001125 0.446518 3 True 15
1 RandomForestMSE_BAG_L2 -53.436895 10.731910 400.680140 0.594810 26.173306 2 True 12
2 ExtraTreesMSE_BAG_L2 -53.840982 10.728724 382.721570 0.591623 8.214736 2 True 14
3 LightGBM_BAG_L2 -55.049296 10.380686 401.045790 0.243585 26.538955 2 True 11
4 CatBoost_BAG_L2 -55.724578 10.220685 444.625357 0.083585 70.118522 2 True 13
5 LightGBMXT_BAG_L2 -60.146446 13.559476 427.593630 3.422376 53.086795 2 True 10
6 KNeighborsDist_BAG_L1 -84.125061 0.104031 0.030274 0.104031 0.030274 1 True 2
7 WeightedEnsemble_L2 -84.125061 0.104782 0.674625 0.000751 0.644350 2 True 9
8 KNeighborsUnif_BAG_L1 -101.546199 0.102817 0.034888 0.102817 0.034888 1 True 1
9 RandomForestMSE_BAG_L1 -116.544294 0.853333 10.812510 0.853333 10.812510 1 True 5
10 ExtraTreesMSE_BAG_L1 -124.588053 0.518733 4.847676 0.518733 4.847676 1 True 7
11 CatBoost_BAG_L1 -130.510645 0.165783 201.262563 0.165783 201.262563 1 True 6
12 LightGBM_BAG_L1 -131.054162 1.275222 27.500071 1.275222 27.500071 1 True 4
13 LightGBMXT_BAG_L1 -131.460909 6.668809 63.862681 6.668809 63.862681 1 True 3
14 NeuralNetFastAI_BAG_L1 -138.033610 0.448373 66.156172 0.448373 66.156172 1 True 8
Number of models trained: 15
Types of models trained:
{'StackerEnsembleModel_RF', 'StackerEnsembleModel_KNN', 'StackerEnsembleModel_XT', 'StackerEnsembleModel_NNFastAiTabular', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_CatBoost', 'WeightedEnsembleModel'}
Bagging used: True (with 8 folds)
Multi-layer stack-ensembling used: True (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 3 | ['season', 'weather', 'humidity']
('int', ['bool']) : 2 | ['holiday', 'workingday']
('int', ['datetime_as_int']) : 5 | ['datetime', 'datetime.year', 'datetime.month', 'datetime.day', 'datetime.dayofweek']
Plot summary of models saved to file: AutogluonModels/ag-20220521_115203/SummaryOfModels.html
*** End of fit() summary ***
Out[15]:
{'model_types': {'KNeighborsUnif_BAG_L1': 'StackerEnsembleModel_KNN',
'KNeighborsDist_BAG_L1': 'StackerEnsembleModel_KNN',
'LightGBMXT_BAG_L1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L1': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L1': 'StackerEnsembleModel_CatBoost',
'ExtraTreesMSE_BAG_L1': 'StackerEnsembleModel_XT',
'NeuralNetFastAI_BAG_L1': 'StackerEnsembleModel_NNFastAiTabular',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBMXT_BAG_L2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L2': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L2': 'StackerEnsembleModel_CatBoost',
'ExtraTreesMSE_BAG_L2': 'StackerEnsembleModel_XT',
'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
'model_performance': {'KNeighborsUnif_BAG_L1': -101.54619908446061,
'KNeighborsDist_BAG_L1': -84.12506123181602,
'LightGBMXT_BAG_L1': -131.46090891834504,
'LightGBM_BAG_L1': -131.054161598899,
'RandomForestMSE_BAG_L1': -116.54429428704391,
'CatBoost_BAG_L1': -130.5106453332399,
'ExtraTreesMSE_BAG_L1': -124.58805258915959,
'NeuralNetFastAI_BAG_L1': -138.03360988203426,
'WeightedEnsemble_L2': -84.12506123181602,
'LightGBMXT_BAG_L2': -60.14644649980745,
'LightGBM_BAG_L2': -55.04929612519262,
'RandomForestMSE_BAG_L2': -53.43689487144225,
'CatBoost_BAG_L2': -55.72457793131616,
'ExtraTreesMSE_BAG_L2': -53.840981843999714,
'WeightedEnsemble_L3': -52.82192379764109},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'KNeighborsUnif_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/KNeighborsUnif_BAG_L1/',
'KNeighborsDist_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/KNeighborsDist_BAG_L1/',
'LightGBMXT_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/LightGBMXT_BAG_L1/',
'LightGBM_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/LightGBM_BAG_L1/',
'RandomForestMSE_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/RandomForestMSE_BAG_L1/',
'CatBoost_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/CatBoost_BAG_L1/',
'ExtraTreesMSE_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/ExtraTreesMSE_BAG_L1/',
'NeuralNetFastAI_BAG_L1': 'AutogluonModels/ag-20220521_115203/models/NeuralNetFastAI_BAG_L1/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_115203/models/WeightedEnsemble_L2/',
'LightGBMXT_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/LightGBMXT_BAG_L2/',
'LightGBM_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/LightGBM_BAG_L2/',
'RandomForestMSE_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/RandomForestMSE_BAG_L2/',
'CatBoost_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/CatBoost_BAG_L2/',
'ExtraTreesMSE_BAG_L2': 'AutogluonModels/ag-20220521_115203/models/ExtraTreesMSE_BAG_L2/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_115203/models/WeightedEnsemble_L3/'},
'model_fit_times': {'KNeighborsUnif_BAG_L1': 0.03488755226135254,
'KNeighborsDist_BAG_L1': 0.030274391174316406,
'LightGBMXT_BAG_L1': 63.86268067359924,
'LightGBM_BAG_L1': 27.500070571899414,
'RandomForestMSE_BAG_L1': 10.81251049041748,
'CatBoost_BAG_L1': 201.26256346702576,
'ExtraTreesMSE_BAG_L1': 4.847675800323486,
'NeuralNetFastAI_BAG_L1': 66.15617179870605,
'WeightedEnsemble_L2': 0.6443502902984619,
'LightGBMXT_BAG_L2': 53.08679533004761,
'LightGBM_BAG_L2': 26.538954973220825,
'RandomForestMSE_BAG_L2': 26.17330551147461,
'CatBoost_BAG_L2': 70.11852192878723,
'ExtraTreesMSE_BAG_L2': 8.214735507965088,
'WeightedEnsemble_L3': 0.4465181827545166},
'model_pred_times': {'KNeighborsUnif_BAG_L1': 0.10281705856323242,
'KNeighborsDist_BAG_L1': 0.10403060913085938,
'LightGBMXT_BAG_L1': 6.668808698654175,
'LightGBM_BAG_L1': 1.2752220630645752,
'RandomForestMSE_BAG_L1': 0.8533329963684082,
'CatBoost_BAG_L1': 0.16578340530395508,
'ExtraTreesMSE_BAG_L1': 0.5187325477600098,
'NeuralNetFastAI_BAG_L1': 0.44837307929992676,
'WeightedEnsemble_L2': 0.0007512569427490234,
'LightGBMXT_BAG_L2': 3.4223759174346924,
'LightGBM_BAG_L2': 0.24358534812927246,
'RandomForestMSE_BAG_L2': 0.5948097705841064,
'CatBoost_BAG_L2': 0.08358502388000488,
'ExtraTreesMSE_BAG_L2': 0.5916233062744141,
'WeightedEnsemble_L3': 0.001125335693359375},
'num_bag_folds': 8,
'max_stack_level': 3,
'model_hyperparams': {'KNeighborsUnif_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'KNeighborsDist_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'LightGBMXT_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'ExtraTreesMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'NeuralNetFastAI_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBMXT_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'ExtraTreesMSE_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -52.821924 11.568244 435.880349
1 RandomForestMSE_BAG_L2 -53.436895 10.731910 400.680140
2 ExtraTreesMSE_BAG_L2 -53.840982 10.728724 382.721570
3 LightGBM_BAG_L2 -55.049296 10.380686 401.045790
4 CatBoost_BAG_L2 -55.724578 10.220685 444.625357
5 LightGBMXT_BAG_L2 -60.146446 13.559476 427.593630
6 KNeighborsDist_BAG_L1 -84.125061 0.104031 0.030274
7 WeightedEnsemble_L2 -84.125061 0.104782 0.674625
8 KNeighborsUnif_BAG_L1 -101.546199 0.102817 0.034888
9 RandomForestMSE_BAG_L1 -116.544294 0.853333 10.812510
10 ExtraTreesMSE_BAG_L1 -124.588053 0.518733 4.847676
11 CatBoost_BAG_L1 -130.510645 0.165783 201.262563
12 LightGBM_BAG_L1 -131.054162 1.275222 27.500071
13 LightGBMXT_BAG_L1 -131.460909 6.668809 63.862681
14 NeuralNetFastAI_BAG_L1 -138.033610 0.448373 66.156172
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.001125 0.446518 3 True
1 0.594810 26.173306 2 True
2 0.591623 8.214736 2 True
3 0.243585 26.538955 2 True
4 0.083585 70.118522 2 True
5 3.422376 53.086795 2 True
6 0.104031 0.030274 1 True
7 0.000751 0.644350 2 True
8 0.102817 0.034888 1 True
9 0.853333 10.812510 1 True
10 0.518733 4.847676 1 True
11 0.165783 201.262563 1 True
12 1.275222 27.500071 1 True
13 6.668809 63.862681 1 True
14 0.448373 66.156172 1 True
fit_order
0 15
1 12
2 14
3 11
4 13
5 10
6 2
7 9
8 1
9 5
10 7
11 6
12 4
13 3
14 8 }
Create predictions from train dataset¶
In [16]:
from sklearn.metrics import mean_squared_error
train_pred = predictor.predict_proba(train)
train_pred_rmse = mean_squared_error(train['count'], train_pred, squared=False)
print('The train rmse is: ',train_pred_rmse)
The train rmse is: 70.86089192948288
Create predictions from test dataset¶
In [17]:
predictions = predictor.predict_proba(test)
predictions.head()
Out[17]:
0 23.889549 1 40.731548 2 44.783401 3 48.364571 4 51.583191 Name: count, dtype: float32
NOTE: Kaggle will reject the submission if we don't set everything to be > 0.¶
In [18]:
# Create submission DataFrame
submission = pd.DataFrame({'datetime': test['datetime'],
'count': predictions})
In [19]:
# Describe the `predictions` series to see if there are any negative values
predictions.describe()
Out[19]:
count 6493.000000 mean 100.680450 std 90.272682 min 2.973643 25% 20.340405 50% 62.326599 75% 170.001556 max 361.611725 Name: count, dtype: float64
In [20]:
# How many negative values do we have?
predictions.isna().sum()
Out[20]:
0
In [21]:
# Set them to zero
import numpy as np
submission.loc[submission['count'] < 0, 'count'] = 0
submission['count'] = submission['count'].fillna(0).astype(int)
In [22]:
# Check submissions
submission.head()
Out[22]:
| datetime | count | |
|---|---|---|
| 0 | 2011-01-20 00:00:00 | 23 |
| 1 | 2011-01-20 01:00:00 | 40 |
| 2 | 2011-01-20 02:00:00 | 44 |
| 3 | 2011-01-20 03:00:00 | 48 |
| 4 | 2011-01-20 04:00:00 | 51 |
Set predictions to submission dataframe, save, and submit¶
In [23]:
# create csv submission file
submission['count'] = submission['count'].astype(np.int64)
submission.to_csv("./submission.csv",index=False)
In [27]:
!kaggle competitions submit -c bike-sharing-demand -f submission.csv -m "first raw submission"
100%|█████████████████████████████████████████| 148k/148k [00:00<00:00, 237kB/s] Successfully submitted to Bike Sharing Demand
View submission via the command line or in the web browser under the competition's page - My Submissions¶
In [28]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore --------------------------- ------------------- --------------------------------- -------- ----------- ------------ submission.csv 2022-05-21 12:08:15 first raw submission complete 1.82081 1.82081 submission_new_hpo.csv 2022-05-19 10:02:34 new features with hyperparameters complete 0.46282 0.46282 submission_new_features.csv 2022-05-19 09:42:12 new features complete 0.46343 0.46343 submission.csv 2022-05-19 09:11:03 first raw submission complete 1.81359 1.81359
Initial score of 1.82081¶
Step 4: Exploratory Data Analysis and Creating an additional feature¶
- Any additional feature will do, but a great suggestion would be to separate out the datetime into hour, day, or month parts.
Data Fields¶
- datetime - hourly date + timestamp
- season - 1 = spring, 2 = summer, 3 = fall, 4 = winter
- holiday - whether the day is considered a holiday
- workingday - whether the day is neither a weekend nor holiday
- weather - 1: Clear, Few clouds, Partly cloudy, Partly cloudy
- 2: Mist + Cloudy, Mist + Broken clouds, Mist + Few clouds, Mist
- 3: Light Snow, Light Rain + Thunderstorm + Scattered clouds, Light Rain + Scattered clouds
- 4: Heavy Rain + Ice Pallets + Thunderstorm + Mist, Snow + Fog
- temp - temperature in Celsius
- atemp - "feels like" temperature in Celsius
- humidity - relative humidity
- windspeed - wind speed
- casual - number of non-registered user rentals initiated
- registered - number of registered user rentals initiated
- count - number of total rentals
In [29]:
import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib import rcParams
%matplotlib inline
In [30]:
# Create a histogram of all features to show the distribution of each one relative to the data. This is part of the exploritory data analysis
train.hist(figsize=(10, 10));
In [31]:
# plot paiplot of train
sns.pairplot(train);
In [32]:
# Plot barplot to see Renting by Weather
rcParams['figure.figsize'] = (5, 5)
sns.set_theme(style="whitegrid")
ax = sns.barplot(data=train,
x='weather',
y='count')
ax.set(title='Sharing by weather')
Out[32]:
[Text(0.5, 1.0, 'Sharing by weather')]
Generaly people prefer in Clear days(1) and Few clouds days(2)
In [33]:
# Plot Boxplot to find outliers in Weather
ax = sns.boxplot(x="weather", y="count", data=train)
In [34]:
# Convert datetime from objects to datetime type
train['datetime'] = pd.to_datetime(train['datetime'])
test['datetime'] = pd.to_datetime(test['datetime'])
# create new features based on datetime
train['year'] = train['datetime'].dt.year
train['month'] = train['datetime'].dt.month
train['day'] = train['datetime'].dt.day
train['hour'] = train['datetime'].dt.hour
test['year'] = test['datetime'].dt.year
test['month'] = test['datetime'].dt.month
test['day'] = test['datetime'].dt.day
test['hour'] = test['datetime'].dt.hour
In [35]:
# Plot the distribution of renting by Hour
rcParams['figure.figsize'] = (15, 8)
fig,ax = plt.subplots()
sns.pointplot(data=train[['hour', 'count', 'season']],
x='hour',
y='count',
hue='season',
ax=ax)
ax.set(title="Season hourly distribution of sharing")
Out[35]:
[Text(0.5, 1.0, 'Season hourly distribution of sharing')]
In [36]:
# Drop datetime
train.drop(train[['datetime']], axis=1, inplace=True)
test.drop(test[['datetime']], axis=1, inplace=True)
In [37]:
# Check correlations after add more features
rcParams['figure.figsize'] = (12, 10)
ax = sns.heatmap(
train.corr(),
cmap="YlGnBu",
square=True,
annot=True
)
Make category types for these so models know they are not just numbers¶
- AutoGluon originally sees these as ints, but in reality they are int representations of a category.
- Setting the dtype to category will classify these as categories in AutoGluon.
In [38]:
# convert to categorical
train["season"] = pd.Categorical(train["season"])
train["weather"] = pd.Categorical(train["weather"])
train["holiday"] = pd.Categorical(train["holiday"])
train["workingday"] = pd.Categorical(train["workingday"])
test["season"] = pd.Categorical(test["season"])
test["weather"] = pd.Categorical(test["weather"])
test["holiday"] = pd.Categorical(test["holiday"])
test["workingday"] = pd.Categorical(test["workingday"])
In [39]:
# View are new feature
train.head()
Out[39]:
| season | holiday | workingday | weather | temp | atemp | humidity | windspeed | casual | registered | count | year | month | day | hour | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 81 | 0.0 | 3 | 13 | 16 | 2011 | 1 | 1 | 0 |
| 1 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 8 | 32 | 40 | 2011 | 1 | 1 | 1 |
| 2 | 1 | 0 | 0 | 1 | 9.02 | 13.635 | 80 | 0.0 | 5 | 27 | 32 | 2011 | 1 | 1 | 2 |
| 3 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 3 | 10 | 13 | 2011 | 1 | 1 | 3 |
| 4 | 1 | 0 | 0 | 1 | 9.84 | 14.395 | 75 | 0.0 | 0 | 1 | 1 | 2011 | 1 | 1 | 4 |
In [40]:
# View histogram of all features again now with the hour feature
train.hist(figsize=(10, 10));
Step 5: Rerun the model with the same settings as before, just with more features¶
In [41]:
predictor_new_features = TabularPredictor(
label='count',
problem_type='regression',
eval_metric='root_mean_squared_error',
learner_kwargs={"ignored_columns": ["casual", "registered"]}
).fit(
train_data=train,
time_limit=900,
presets='best_quality'
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_121113/"
Presets specified: ['best_quality']
Beginning AutoGluon training ... Time limit = 900s
AutoGluon will save models to "AutogluonModels/ag-20220521_121113/"
AutoGluon Version: 0.4.1
Python Version: 3.7.10
Operating System: Linux
Train Data Rows: 10886
Train Data Columns: 14
Label Column: count
Preprocessing data ...
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 1794.16 MB
Train Data (Original) Memory Usage: 0.74 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('category', []) : 4 | ['season', 'holiday', 'workingday', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 5 | ['humidity', 'year', 'month', 'day', 'hour']
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
0.1s = Fit runtime
12 features in original data used to generate 12 features in processed data.
Train Data (Processed) Memory Usage: 0.67 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.19s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 11 L1 models ...
Fitting model: KNeighborsUnif_BAG_L1 ... Training model for up to 599.72s of the 899.79s of remaining time.
-123.9333 = Validation score (root_mean_squared_error)
0.03s = Training runtime
0.2s = Validation runtime
Fitting model: KNeighborsDist_BAG_L1 ... Training model for up to 599.24s of the 899.31s of remaining time.
-119.3656 = Validation score (root_mean_squared_error)
0.03s = Training runtime
0.21s = Validation runtime
Fitting model: LightGBMXT_BAG_L1 ... Training model for up to 598.76s of the 898.83s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-37.0978 = Validation score (root_mean_squared_error)
93.72s = Training runtime
17.59s = Validation runtime
Fitting model: LightGBM_BAG_L1 ... Training model for up to 499.28s of the 799.35s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-37.6929 = Validation score (root_mean_squared_error)
44.41s = Training runtime
3.65s = Validation runtime
Fitting model: RandomForestMSE_BAG_L1 ... Training model for up to 450.32s of the 750.4s of remaining time.
-42.1495 = Validation score (root_mean_squared_error)
9.36s = Training runtime
0.57s = Validation runtime
Fitting model: CatBoost_BAG_L1 ... Training model for up to 437.79s of the 737.86s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-37.8729 = Validation score (root_mean_squared_error)
360.07s = Training runtime
0.34s = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L1 ... Training model for up to 74.82s of the 374.89s of remaining time.
-41.6234 = Validation score (root_mean_squared_error)
5.78s = Training runtime
0.55s = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L1 ... Training model for up to 65.9s of the 365.98s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-55.7648 = Validation score (root_mean_squared_error)
73.39s = Training runtime
0.53s = Validation runtime
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 289.56s of remaining time.
-35.8693 = Validation score (root_mean_squared_error)
1.03s = Training runtime
0.0s = Validation runtime
Fitting 9 L2 models ...
Fitting model: LightGBMXT_BAG_L2 ... Training model for up to 288.44s of the 288.42s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.9157 = Validation score (root_mean_squared_error)
20.56s = Training runtime
0.29s = Validation runtime
Fitting model: LightGBM_BAG_L2 ... Training model for up to 265.26s of the 265.24s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.1403 = Validation score (root_mean_squared_error)
20.15s = Training runtime
0.15s = Validation runtime
Fitting model: RandomForestMSE_BAG_L2 ... Training model for up to 242.44s of the 242.41s of remaining time.
-36.661 = Validation score (root_mean_squared_error)
26.85s = Training runtime
0.61s = Validation runtime
Fitting model: CatBoost_BAG_L2 ... Training model for up to 212.57s of the 212.55s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.1668 = Validation score (root_mean_squared_error)
54.01s = Training runtime
0.12s = Validation runtime
Fitting model: ExtraTreesMSE_BAG_L2 ... Training model for up to 155.9s of the 155.88s of remaining time.
-36.1166 = Validation score (root_mean_squared_error)
7.95s = Training runtime
0.59s = Validation runtime
Fitting model: NeuralNetFastAI_BAG_L2 ... Training model for up to 144.8s of the 144.78s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.318 = Validation score (root_mean_squared_error)
107.9s = Training runtime
0.54s = Validation runtime
Fitting model: XGBoost_BAG_L2 ... Training model for up to 33.88s of the 33.86s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
-36.5113 = Validation score (root_mean_squared_error)
23.33s = Training runtime
0.15s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2 ... Training model for up to 7.48s of the 7.46s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
Warning: Exception caused NeuralNetTorch_BAG_L2 to fail during training... Skipping this model.
ray::_ray_fit() (pid=4801, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 289, in _ray_fit
time_limit=time_limit_fold, num_cpus=num_cpus, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 579, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 203, in _fit
**fit_kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 377, in _train_net
raise AssertionError('0 epochs trained!')
AssertionError: 0 epochs trained!
Detailed Traceback:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/autogluon/core/trainer/abstract_trainer.py", line 1074, in _train_and_save
model = self._train_single(X, y, model, X_val, y_val, **model_fit_kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/trainer/abstract_trainer.py", line 1032, in _train_single
model = model.fit(X=X, y=y, X_val=X_val, y_val=y_val, **model_fit_kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 579, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/stacker_ensemble_model.py", line 153, in _fit
return super()._fit(X=X, y=y, time_limit=time_limit, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 234, in _fit
n_repeats=n_repeats, n_repeat_start=n_repeat_start, save_folds=save_bag_folds, groups=groups, **kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/bagged_ensemble_model.py", line 502, in _fit_folds
fold_fitting_strategy.after_all_folds_scheduled()
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 455, in after_all_folds_scheduled
raise processed_exception
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 423, in after_all_folds_scheduled
time_end_fit, predict_time, predict_1_time = self.ray.get(finished)
File "/usr/local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/ray/worker.py", line 1733, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AssertionError): ray::_ray_fit() (pid=4801, ip=169.255.254.2)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/ensemble/fold_fitting_strategy.py", line 289, in _ray_fit
time_limit=time_limit_fold, num_cpus=num_cpus, **kwargs_fold)
File "/usr/local/lib/python3.7/site-packages/autogluon/core/models/abstract/abstract_model.py", line 579, in fit
out = self._fit(**kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 203, in _fit
**fit_kwargs)
File "/usr/local/lib/python3.7/site-packages/autogluon/tabular/models/tabular_nn/torch/tabular_nn_torch.py", line 377, in _train_net
raise AssertionError('0 epochs trained!')
AssertionError: 0 epochs trained!
Fitting model: LightGBMLarge_BAG_L2 ... Training model for up to 0.38s of the 0.36s of remaining time.
Fitting 8 child models (S1F1 - S1F8) | Fitting with ParallelLocalFoldFittingStrategy
2022-05-21 12:26:14,257 WARNING services.py:1866 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This will harm performance! You may be able to free up space by deleting files in /dev/shm. If you are inside a Docker container, you can increase /dev/shm size by passing '--shm-size=0.73gb' to 'docker run' (or add it to the run_options list in a Ray cluster config). Make sure to set this to more than 30% of available RAM.
Time limit exceeded... Skipping LightGBMLarge_BAG_L2.
Completed 1/20 k-fold bagging repeats ...
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the -3.66s of remaining time.
-35.7039 = Validation score (root_mean_squared_error)
0.49s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 904.38s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_121113/")
In [42]:
predictor_new_features.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -35.703944 25.801695 827.468261 0.001348 0.487528 3 True 17
1 WeightedEnsemble_L2 -35.869295 22.151278 508.585230 0.000794 1.028767 2 True 9
2 ExtraTreesMSE_BAG_L2 -36.116601 24.237407 594.732446 0.592123 7.949546 2 True 14
3 LightGBM_BAG_L2 -36.140333 23.796642 606.932092 0.151358 20.149193 2 True 11
4 CatBoost_BAG_L2 -36.166801 23.760818 640.795617 0.115534 54.012717 2 True 13
5 NeuralNetFastAI_BAG_L2 -36.318005 24.180289 694.683991 0.535005 107.901092 2 True 15
6 XGBoost_BAG_L2 -36.511258 23.794355 610.117719 0.149071 23.334819 2 True 16
7 RandomForestMSE_BAG_L2 -36.661025 24.257257 613.633366 0.611973 26.850466 2 True 12
8 LightGBMXT_BAG_L2 -36.915742 23.938235 607.344602 0.292951 20.561702 2 True 10
9 LightGBMXT_BAG_L1 -37.097828 17.588601 93.720337 17.588601 93.720337 1 True 3
10 LightGBM_BAG_L1 -37.692850 3.653144 44.405111 3.653144 44.405111 1 True 4
11 CatBoost_BAG_L1 -37.872933 0.342373 360.073151 0.342373 360.073151 1 True 6
12 ExtraTreesMSE_BAG_L1 -41.623410 0.550196 5.776536 0.550196 5.776536 1 True 7
13 RandomForestMSE_BAG_L1 -42.149545 0.566365 9.357864 0.566365 9.357864 1 True 5
14 NeuralNetFastAI_BAG_L1 -55.764827 0.534983 73.393159 0.534983 73.393159 1 True 8
15 KNeighborsDist_BAG_L1 -119.365601 0.206364 0.027186 0.206364 0.027186 1 True 2
16 KNeighborsUnif_BAG_L1 -123.933260 0.203257 0.029556 0.203257 0.029556 1 True 1
Number of models trained: 17
Types of models trained:
{'StackerEnsembleModel_RF', 'StackerEnsembleModel_KNN', 'StackerEnsembleModel_XT', 'StackerEnsembleModel_NNFastAiTabular', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_XGBoost', 'StackerEnsembleModel_CatBoost', 'WeightedEnsembleModel'}
Bagging used: True (with 8 folds)
Multi-layer stack-ensembling used: True (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
Plot summary of models saved to file: AutogluonModels/ag-20220521_121113/SummaryOfModels.html
*** End of fit() summary ***
Out[42]:
{'model_types': {'KNeighborsUnif_BAG_L1': 'StackerEnsembleModel_KNN',
'KNeighborsDist_BAG_L1': 'StackerEnsembleModel_KNN',
'LightGBMXT_BAG_L1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L1': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L1': 'StackerEnsembleModel_CatBoost',
'ExtraTreesMSE_BAG_L1': 'StackerEnsembleModel_XT',
'NeuralNetFastAI_BAG_L1': 'StackerEnsembleModel_NNFastAiTabular',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBMXT_BAG_L2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2': 'StackerEnsembleModel_LGB',
'RandomForestMSE_BAG_L2': 'StackerEnsembleModel_RF',
'CatBoost_BAG_L2': 'StackerEnsembleModel_CatBoost',
'ExtraTreesMSE_BAG_L2': 'StackerEnsembleModel_XT',
'NeuralNetFastAI_BAG_L2': 'StackerEnsembleModel_NNFastAiTabular',
'XGBoost_BAG_L2': 'StackerEnsembleModel_XGBoost',
'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
'model_performance': {'KNeighborsUnif_BAG_L1': -123.93326033133259,
'KNeighborsDist_BAG_L1': -119.36560130583281,
'LightGBMXT_BAG_L1': -37.09782821469706,
'LightGBM_BAG_L1': -37.69285008347106,
'RandomForestMSE_BAG_L1': -42.14954493458521,
'CatBoost_BAG_L1': -37.87293345815347,
'ExtraTreesMSE_BAG_L1': -41.623410397279095,
'NeuralNetFastAI_BAG_L1': -55.764827348159514,
'WeightedEnsemble_L2': -35.86929496695115,
'LightGBMXT_BAG_L2': -36.915742419191254,
'LightGBM_BAG_L2': -36.14033348245729,
'RandomForestMSE_BAG_L2': -36.66102537394624,
'CatBoost_BAG_L2': -36.16680134531193,
'ExtraTreesMSE_BAG_L2': -36.11660135289378,
'NeuralNetFastAI_BAG_L2': -36.31800497190385,
'XGBoost_BAG_L2': -36.51125766145703,
'WeightedEnsemble_L3': -35.70394398419218},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'KNeighborsUnif_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/KNeighborsUnif_BAG_L1/',
'KNeighborsDist_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/KNeighborsDist_BAG_L1/',
'LightGBMXT_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/LightGBMXT_BAG_L1/',
'LightGBM_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/LightGBM_BAG_L1/',
'RandomForestMSE_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/RandomForestMSE_BAG_L1/',
'CatBoost_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/CatBoost_BAG_L1/',
'ExtraTreesMSE_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/ExtraTreesMSE_BAG_L1/',
'NeuralNetFastAI_BAG_L1': 'AutogluonModels/ag-20220521_121113/models/NeuralNetFastAI_BAG_L1/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_121113/models/WeightedEnsemble_L2/',
'LightGBMXT_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/LightGBMXT_BAG_L2/',
'LightGBM_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/LightGBM_BAG_L2/',
'RandomForestMSE_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/RandomForestMSE_BAG_L2/',
'CatBoost_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/CatBoost_BAG_L2/',
'ExtraTreesMSE_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/ExtraTreesMSE_BAG_L2/',
'NeuralNetFastAI_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/NeuralNetFastAI_BAG_L2/',
'XGBoost_BAG_L2': 'AutogluonModels/ag-20220521_121113/models/XGBoost_BAG_L2/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_121113/models/WeightedEnsemble_L3/'},
'model_fit_times': {'KNeighborsUnif_BAG_L1': 0.029555797576904297,
'KNeighborsDist_BAG_L1': 0.02718639373779297,
'LightGBMXT_BAG_L1': 93.72033739089966,
'LightGBM_BAG_L1': 44.40511083602905,
'RandomForestMSE_BAG_L1': 9.357863903045654,
'CatBoost_BAG_L1': 360.0731511116028,
'ExtraTreesMSE_BAG_L1': 5.776535511016846,
'NeuralNetFastAI_BAG_L1': 73.39315891265869,
'WeightedEnsemble_L2': 1.0287666320800781,
'LightGBMXT_BAG_L2': 20.561702489852905,
'LightGBM_BAG_L2': 20.149192571640015,
'RandomForestMSE_BAG_L2': 26.85046625137329,
'CatBoost_BAG_L2': 54.01271724700928,
'ExtraTreesMSE_BAG_L2': 7.9495463371276855,
'NeuralNetFastAI_BAG_L2': 107.90109157562256,
'XGBoost_BAG_L2': 23.334819078445435,
'WeightedEnsemble_L3': 0.48752784729003906},
'model_pred_times': {'KNeighborsUnif_BAG_L1': 0.20325732231140137,
'KNeighborsDist_BAG_L1': 0.20636367797851562,
'LightGBMXT_BAG_L1': 17.5886013507843,
'LightGBM_BAG_L1': 3.653143882751465,
'RandomForestMSE_BAG_L1': 0.5663654804229736,
'CatBoost_BAG_L1': 0.3423731327056885,
'ExtraTreesMSE_BAG_L1': 0.5501961708068848,
'NeuralNetFastAI_BAG_L1': 0.5349831581115723,
'WeightedEnsemble_L2': 0.0007936954498291016,
'LightGBMXT_BAG_L2': 0.2929508686065674,
'LightGBM_BAG_L2': 0.15135788917541504,
'RandomForestMSE_BAG_L2': 0.6119728088378906,
'CatBoost_BAG_L2': 0.11553406715393066,
'ExtraTreesMSE_BAG_L2': 0.5921225547790527,
'NeuralNetFastAI_BAG_L2': 0.5350046157836914,
'XGBoost_BAG_L2': 0.14907097816467285,
'WeightedEnsemble_L3': 0.001348257064819336},
'num_bag_folds': 8,
'max_stack_level': 3,
'model_hyperparams': {'KNeighborsUnif_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'KNeighborsDist_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'LightGBMXT_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'ExtraTreesMSE_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'NeuralNetFastAI_BAG_L1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBMXT_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'RandomForestMSE_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'CatBoost_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'ExtraTreesMSE_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True,
'use_child_oof': True},
'NeuralNetFastAI_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'XGBoost_BAG_L2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -35.703944 25.801695 827.468261
1 WeightedEnsemble_L2 -35.869295 22.151278 508.585230
2 ExtraTreesMSE_BAG_L2 -36.116601 24.237407 594.732446
3 LightGBM_BAG_L2 -36.140333 23.796642 606.932092
4 CatBoost_BAG_L2 -36.166801 23.760818 640.795617
5 NeuralNetFastAI_BAG_L2 -36.318005 24.180289 694.683991
6 XGBoost_BAG_L2 -36.511258 23.794355 610.117719
7 RandomForestMSE_BAG_L2 -36.661025 24.257257 613.633366
8 LightGBMXT_BAG_L2 -36.915742 23.938235 607.344602
9 LightGBMXT_BAG_L1 -37.097828 17.588601 93.720337
10 LightGBM_BAG_L1 -37.692850 3.653144 44.405111
11 CatBoost_BAG_L1 -37.872933 0.342373 360.073151
12 ExtraTreesMSE_BAG_L1 -41.623410 0.550196 5.776536
13 RandomForestMSE_BAG_L1 -42.149545 0.566365 9.357864
14 NeuralNetFastAI_BAG_L1 -55.764827 0.534983 73.393159
15 KNeighborsDist_BAG_L1 -119.365601 0.206364 0.027186
16 KNeighborsUnif_BAG_L1 -123.933260 0.203257 0.029556
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.001348 0.487528 3 True
1 0.000794 1.028767 2 True
2 0.592123 7.949546 2 True
3 0.151358 20.149193 2 True
4 0.115534 54.012717 2 True
5 0.535005 107.901092 2 True
6 0.149071 23.334819 2 True
7 0.611973 26.850466 2 True
8 0.292951 20.561702 2 True
9 17.588601 93.720337 1 True
10 3.653144 44.405111 1 True
11 0.342373 360.073151 1 True
12 0.550196 5.776536 1 True
13 0.566365 9.357864 1 True
14 0.534983 73.393159 1 True
15 0.206364 0.027186 1 True
16 0.203257 0.029556 1 True
fit_order
0 17
1 9
2 14
3 11
4 13
5 15
6 16
7 12
8 10
9 3
10 4
11 6
12 7
13 5
14 8
15 2
16 1 }
Create predictions for train dataset¶
In [43]:
# predictor_new_features for training
predictions_new_features_train = predictor_new_features.predict_proba(train)
rmse_new_features = mean_squared_error(train['count'], predictions_new_features_train, squared=False)
print('The rmse for new feateares in train is: ',rmse_new_features)
The rmse for new feateares in train is: 18.306591272491698
Create predictions for test dataset
In [44]:
#predictor_new_features for testing
predictions_new_features = predictor_new_features.predict_proba(test)
In [45]:
# Remember to set all negative values to zero
submission_new_features = pd.DataFrame({'datetime': submission['datetime'],
'count': predictions_new_features})
submission_new_features.loc[submission_new_features['count'] <0, 'count'] = 0
In [46]:
# Same submitting predictions
submission_new_features['count'] = submission_new_features['count'].fillna(0).astype(int)
submission_new_features['count'] = submission_new_features['count'].astype(np.int64)
submission_new_features.to_csv("submission_new_features.csv", index=False)
In [47]:
!kaggle competitions submit -c bike-sharing-demand -f submission_new_features.csv -m "new features"
100%|█████████████████████████████████████████| 149k/149k [00:00<00:00, 256kB/s] Successfully submitted to Bike Sharing Demand
In [48]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore --------------------------- ------------------- --------------------------------- -------- ----------- ------------ submission_new_features.csv 2022-05-21 12:28:51 new features complete 0.46603 0.46603 submission.csv 2022-05-21 12:08:15 first raw submission complete 1.82081 1.82081 submission_new_hpo.csv 2022-05-19 10:02:34 new features with hyperparameters complete 0.46282 0.46282 submission_new_features.csv 2022-05-19 09:42:12 new features complete 0.46343 0.46343
New Score of 0.46603¶
Step 6: Hyper parameter optimization¶
- There are many options for hyper parameter optimization.
- Options are to change the AutoGluon higher level parameters or the individual model hyperparameters.
- The hyperparameters of the models themselves that are in AutoGluon. Those need the
hyperparameterandhyperparameter_tune_kwargsarguments.
In [51]:
# Defining hyperparameter and hyperparameter_tune_kwargs
import autogluon.core as ag
nn_options = {
'num_epochs': 10,
'learning_rate': ag.space.Real(1e-4, 1e-2, default=5e-4, log=True),
'activation': ag.space.Categorical('relu', 'softrelu', 'tanh'),
'dropout_prob': ag.space.Real(0.0, 0.5, default=0.1),
}
gbm_options = {
'num_boost_round': 100,
'num_leaves': ag.space.Int(lower=26, upper=66, default=36),
}
hyperparameters = {
'GBM': gbm_options,
'NN_TORCH': nn_options,
}
hyperparameter_tune_kwargs = {
'num_trials': 5,
'scheduler' : 'local',
'searcher': 'auto',
}
In [52]:
predictor_new_hpo = TabularPredictor(
label="count",
eval_metric='root_mean_squared_error',
learner_kwargs={'ignored_columns': ["casual", "registered"]}
).fit(
train_data=train,
time_limit=1200,
num_bag_folds=5,
num_bag_sets=1,
num_stack_levels=1,
presets="best_quality",
hyperparameters=hyperparameters,
hyperparameter_tune_kwargs=hyperparameter_tune_kwargs,
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_123126/"
Presets specified: ['best_quality']
Warning: hyperparameter tuning is currently experimental and may cause the process to hang.
Beginning AutoGluon training ... Time limit = 1200s
AutoGluon will save models to "AutogluonModels/ag-20220521_123126/"
AutoGluon Version: 0.4.1
Python Version: 3.7.10
Operating System: Linux
Train Data Rows: 10886
Train Data Columns: 14
Label Column: count
Preprocessing data ...
AutoGluon infers your prediction problem is: 'regression' (because dtype of label-column == int and many unique label-values observed).
Label info (max, min, mean, stddev): (977, 1, 191.57413, 181.14445)
If 'regression' is not the correct problem_type, please manually specify the problem_type parameter during predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression'])
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 1873.43 MB
Train Data (Original) Memory Usage: 0.74 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('category', []) : 4 | ['season', 'holiday', 'workingday', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 5 | ['humidity', 'year', 'month', 'day', 'hour']
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
0.1s = Fit runtime
12 features in original data used to generate 12 features in processed data.
Train Data (Processed) Memory Usage: 0.67 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.18s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 2 stack levels (L1 to L2) ...
Fitting 2 L1 models ...
Hyperparameter tuning model: LightGBM_BAG_L1 ... Tuning model for up to 71.97s of the 1199.82s of remaining time.
100%|██████████| 5/5 [00:02<00:00, 2.37it/s]
Fitted model: LightGBM_BAG_L1/T1 ...
-45.1728 = Validation score (root_mean_squared_error)
0.38s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L1/T2 ...
-42.4191 = Validation score (root_mean_squared_error)
0.31s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L1/T3 ...
-44.8062 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L1/T4 ...
-126.8104 = Validation score (root_mean_squared_error)
0.3s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T5 ...
-48.5942 = Validation score (root_mean_squared_error)
0.34s = Training runtime
0.02s = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L1 ... Tuning model for up to 71.97s of the 1196.89s of remaining time.
80%|████████ | 4/5 [00:37<00:10, 10.73s/it] Stopping HPO to satisfy time limit...
80%|████████ | 4/5 [00:56<00:14, 14.16s/it]
Fitted model: NeuralNetTorch_BAG_L1/T1 ...
-102.1329 = Validation score (root_mean_squared_error)
5.42s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T2 ...
-58.9307 = Validation score (root_mean_squared_error)
8.91s = Training runtime
0.05s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T3 ...
-84.9915 = Validation score (root_mean_squared_error)
8.0s = Training runtime
0.05s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T4 ...
-56.5326 = Validation score (root_mean_squared_error)
14.93s = Training runtime
0.06s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T5 ...
-145.217 = Validation score (root_mean_squared_error)
18.51s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 738.55s of the 1138.69s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-44.719 = Validation score (root_mean_squared_error)
7.46s = Training runtime
0.11s = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 729.19s of the 1129.32s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-42.8473 = Validation score (root_mean_squared_error)
6.55s = Training runtime
0.13s = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 720.36s of the 1120.49s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-43.6524 = Validation score (root_mean_squared_error)
6.9s = Training runtime
0.14s = Validation runtime
Fitting model: LightGBM_BAG_L1/T4 ... Training model for up to 711.2s of the 1111.34s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-124.3498 = Validation score (root_mean_squared_error)
6.9s = Training runtime
0.09s = Validation runtime
Fitting model: LightGBM_BAG_L1/T5 ... Training model for up to 701.92s of the 1102.05s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
2022-05-21 12:33:06,004 WARNING worker.py:1257 -- The actor or task with ID 275e16fd1756370196a0a6166725870344cbe5fe01000000 cannot be scheduled right now. You can ignore this message if this Ray cluster is expected to auto-scale or if you specified a runtime_env for this actor or task, which may take time to install. Otherwise, this is likely due to all cluster resources being claimed by actors. To resolve the issue, consider creating fewer actors or increasing the resources available to this Ray cluster.
Required resources for this actor or task: {CPU: 1.000000}
Available resources on this node: {1.000000/2.000000 CPU, 1.319749 GiB/1.319749 GiB memory, 0.659874 GiB/0.659874 GiB object_store_memory, 1.000000/1.000000 node:169.255.254.2}
In total there are 2 pending tasks and 0 pending actors on this node.
-47.8441 = Validation score (root_mean_squared_error)
7.73s = Training runtime
0.12s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T1 ... Training model for up to 691.07s of the 1091.2s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-96.49 = Validation score (root_mean_squared_error)
25.51s = Training runtime
0.39s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T2 ... Training model for up to 667.71s of the 1067.85s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-56.923 = Validation score (root_mean_squared_error)
38.11s = Training runtime
0.31s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T3 ... Training model for up to 635.55s of the 1035.68s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-81.996 = Validation score (root_mean_squared_error)
35.47s = Training runtime
0.34s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T4 ... Training model for up to 604.4s of the 1004.54s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-53.8145 = Validation score (root_mean_squared_error)
72.15s = Training runtime
0.43s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L1/T5 ... Training model for up to 544.51s of the 944.64s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-140.6635 = Validation score (root_mean_squared_error)
88.63s = Training runtime
0.49s = Validation runtime
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 871.12s of remaining time.
-42.2555 = Validation score (root_mean_squared_error)
0.8s = Training runtime
0.0s = Validation runtime
Fitting 2 L2 models ...
Hyperparameter tuning model: LightGBM_BAG_L2 ... Tuning model for up to 78.32s of the 870.22s of remaining time.
100%|██████████| 5/5 [00:02<00:00, 1.80it/s]
Fitted model: LightGBM_BAG_L2/T1 ...
-43.6231 = Validation score (root_mean_squared_error)
0.46s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L2/T2 ...
-43.3439 = Validation score (root_mean_squared_error)
0.42s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T3 ...
-43.7476 = Validation score (root_mean_squared_error)
0.58s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L2/T4 ...
-105.803 = Validation score (root_mean_squared_error)
0.43s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T5 ...
-43.9841 = Validation score (root_mean_squared_error)
0.51s = Training runtime
0.01s = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L2 ... Tuning model for up to 78.32s of the 866.66s of remaining time.
100%|██████████| 5/5 [00:48<00:00, 9.75s/it]
Fitted model: NeuralNetTorch_BAG_L2/T1 ...
-45.5715 = Validation score (root_mean_squared_error)
5.24s = Training runtime
0.05s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T2 ...
-45.7959 = Validation score (root_mean_squared_error)
7.09s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T3 ...
-48.7631 = Validation score (root_mean_squared_error)
6.34s = Training runtime
0.06s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T4 ...
-44.9841 = Validation score (root_mean_squared_error)
12.4s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T5 ...
-47.2174 = Validation score (root_mean_squared_error)
16.73s = Training runtime
0.08s = Validation runtime
Fitting model: LightGBM_BAG_L2/T1 ... Training model for up to 816.42s of the 816.4s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-41.2004 = Validation score (root_mean_squared_error)
7.92s = Training runtime
0.1s = Validation runtime
Fitting model: LightGBM_BAG_L2/T2 ... Training model for up to 805.65s of the 805.63s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-41.0374 = Validation score (root_mean_squared_error)
7.11s = Training runtime
0.1s = Validation runtime
Fitting model: LightGBM_BAG_L2/T3 ... Training model for up to 796.0s of the 795.98s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-41.3462 = Validation score (root_mean_squared_error)
7.99s = Training runtime
0.14s = Validation runtime
Fitting model: LightGBM_BAG_L2/T4 ... Training model for up to 784.58s of the 784.55s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-104.121 = Validation score (root_mean_squared_error)
7.18s = Training runtime
0.08s = Validation runtime
Fitting model: LightGBM_BAG_L2/T5 ... Training model for up to 775.08s of the 775.05s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-41.6475 = Validation score (root_mean_squared_error)
7.91s = Training runtime
0.11s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T1 ... Training model for up to 764.83s of the 764.81s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-42.5771 = Validation score (root_mean_squared_error)
25.69s = Training runtime
0.33s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T2 ... Training model for up to 740.9s of the 740.88s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-42.6448 = Validation score (root_mean_squared_error)
36.17s = Training runtime
0.34s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T3 ... Training model for up to 708.82s of the 708.8s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-45.6104 = Validation score (root_mean_squared_error)
35.41s = Training runtime
0.42s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T4 ... Training model for up to 676.28s of the 676.26s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-42.5829 = Validation score (root_mean_squared_error)
60.85s = Training runtime
0.6s = Validation runtime
Fitting model: NeuralNetTorch_BAG_L2/T5 ... Training model for up to 624.87s of the 624.85s of remaining time.
Fitting 4 child models (S1F2 - S1F5) | Fitting with ParallelLocalFoldFittingStrategy
-43.8804 = Validation score (root_mean_squared_error)
81.01s = Training runtime
0.72s = Validation runtime
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the 556.88s of remaining time.
-40.7067 = Validation score (root_mean_squared_error)
0.87s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 644.24s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_123126/")
In [53]:
predictor_new_hpo.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -40.706686 5.304729 558.431824 0.001316 0.874229 3 True 22
1 LightGBM_BAG_L2/T2 -41.037416 2.635418 302.515243 0.095234 7.105219 2 True 13
2 LightGBM_BAG_L2/T1 -41.200410 2.642720 303.331255 0.102536 7.921231 2 True 12
3 LightGBM_BAG_L2/T3 -41.346167 2.680607 303.401063 0.140424 7.991039 2 True 14
4 LightGBM_BAG_L2/T5 -41.647512 2.650338 303.322842 0.110154 7.912818 2 True 16
5 WeightedEnsemble_L2 -42.255533 1.008055 124.509622 0.000832 0.797046 2 True 11
6 NeuralNetTorch_BAG_L2/T1 -42.577094 2.868365 321.103085 0.328181 25.693061 2 True 17
7 NeuralNetTorch_BAG_L2/T4 -42.582856 3.144919 356.262383 0.604735 60.852359 2 True 20
8 NeuralNetTorch_BAG_L2/T2 -42.644782 2.884022 331.575251 0.343839 36.165227 2 True 18
9 LightGBM_BAG_L1/T2 -42.847297 0.129618 6.548905 0.129618 6.548905 1 True 2
10 LightGBM_BAG_L1/T3 -43.652435 0.135641 6.900732 0.135641 6.900732 1 True 3
11 NeuralNetTorch_BAG_L2/T5 -43.880374 3.265158 376.423512 0.724974 81.013488 2 True 21
12 LightGBM_BAG_L1/T1 -44.719007 0.110728 7.457866 0.110728 7.457866 1 True 1
13 NeuralNetTorch_BAG_L2/T3 -45.610407 2.963489 330.815973 0.423306 35.405949 2 True 19
14 LightGBM_BAG_L1/T5 -47.844107 0.117313 7.732576 0.117313 7.732576 1 True 5
15 NeuralNetTorch_BAG_L1/T4 -53.814540 0.427469 72.154497 0.427469 72.154497 1 True 9
16 NeuralNetTorch_BAG_L1/T2 -56.923024 0.314495 38.108442 0.314495 38.108442 1 True 7
17 NeuralNetTorch_BAG_L1/T3 -81.995966 0.336665 35.471487 0.336665 35.471487 1 True 8
18 NeuralNetTorch_BAG_L1/T1 -96.490001 0.385836 25.508792 0.385836 25.508792 1 True 6
19 LightGBM_BAG_L2/T4 -104.120960 2.621153 302.590129 0.080969 7.180105 2 True 15
20 LightGBM_BAG_L1/T4 -124.349788 0.094529 6.897573 0.094529 6.897573 1 True 4
21 NeuralNetTorch_BAG_L1/T5 -140.663465 0.487890 88.629153 0.487890 88.629153 1 True 10
Number of models trained: 22
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_TabularNeuralNetTorch'}
Bagging used: True (with 5 folds)
Multi-layer stack-ensembling used: True (with 3 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
Plot summary of models saved to file: AutogluonModels/ag-20220521_123126/SummaryOfModels.html
*** End of fit() summary ***
Out[53]:
{'model_types': {'LightGBM_BAG_L1/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T3': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T4': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T5': 'StackerEnsembleModel_LGB',
'NeuralNetTorch_BAG_L1/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBM_BAG_L2/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T3': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T4': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T5': 'StackerEnsembleModel_LGB',
'NeuralNetTorch_BAG_L2/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
'WeightedEnsemble_L3': 'WeightedEnsembleModel'},
'model_performance': {'LightGBM_BAG_L1/T1': -44.719006568986714,
'LightGBM_BAG_L1/T2': -42.84729671304735,
'LightGBM_BAG_L1/T3': -43.65243479420564,
'LightGBM_BAG_L1/T4': -124.34978848933305,
'LightGBM_BAG_L1/T5': -47.844107439841125,
'NeuralNetTorch_BAG_L1/T1': -96.49000098962254,
'NeuralNetTorch_BAG_L1/T2': -56.923023716520504,
'NeuralNetTorch_BAG_L1/T3': -81.995966464309,
'NeuralNetTorch_BAG_L1/T4': -53.814540417474646,
'NeuralNetTorch_BAG_L1/T5': -140.66346547927142,
'WeightedEnsemble_L2': -42.255532640311394,
'LightGBM_BAG_L2/T1': -41.20040982056577,
'LightGBM_BAG_L2/T2': -41.03741568144702,
'LightGBM_BAG_L2/T3': -41.34616693698568,
'LightGBM_BAG_L2/T4': -104.12096022803408,
'LightGBM_BAG_L2/T5': -41.64751175358479,
'NeuralNetTorch_BAG_L2/T1': -42.577093629311705,
'NeuralNetTorch_BAG_L2/T2': -42.64478220404302,
'NeuralNetTorch_BAG_L2/T3': -45.61040725200952,
'NeuralNetTorch_BAG_L2/T4': -42.58285563687108,
'NeuralNetTorch_BAG_L2/T5': -43.88037422572748,
'WeightedEnsemble_L3': -40.70668617525216},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'LightGBM_BAG_L1/T1': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T1/',
'LightGBM_BAG_L1/T2': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T2/',
'LightGBM_BAG_L1/T3': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T3/',
'LightGBM_BAG_L1/T4': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T4/',
'LightGBM_BAG_L1/T5': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L1/T5/',
'NeuralNetTorch_BAG_L1/T1': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T1/',
'NeuralNetTorch_BAG_L1/T2': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T2/',
'NeuralNetTorch_BAG_L1/T3': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T3/',
'NeuralNetTorch_BAG_L1/T4': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T4/',
'NeuralNetTorch_BAG_L1/T5': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L1/T5/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_123126/models/WeightedEnsemble_L2/',
'LightGBM_BAG_L2/T1': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T1/',
'LightGBM_BAG_L2/T2': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T2/',
'LightGBM_BAG_L2/T3': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T3/',
'LightGBM_BAG_L2/T4': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T4/',
'LightGBM_BAG_L2/T5': 'AutogluonModels/ag-20220521_123126/models/LightGBM_BAG_L2/T5/',
'NeuralNetTorch_BAG_L2/T1': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T1/',
'NeuralNetTorch_BAG_L2/T2': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T2/',
'NeuralNetTorch_BAG_L2/T3': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T3/',
'NeuralNetTorch_BAG_L2/T4': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T4/',
'NeuralNetTorch_BAG_L2/T5': 'AutogluonModels/ag-20220521_123126/models/NeuralNetTorch_BAG_L2/T5/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_123126/models/WeightedEnsemble_L3/'},
'model_fit_times': {'LightGBM_BAG_L1/T1': 7.457866430282593,
'LightGBM_BAG_L1/T2': 6.54890513420105,
'LightGBM_BAG_L1/T3': 6.900731563568115,
'LightGBM_BAG_L1/T4': 6.897572994232178,
'LightGBM_BAG_L1/T5': 7.732576370239258,
'NeuralNetTorch_BAG_L1/T1': 25.508792400360107,
'NeuralNetTorch_BAG_L1/T2': 38.108442068099976,
'NeuralNetTorch_BAG_L1/T3': 35.47148656845093,
'NeuralNetTorch_BAG_L1/T4': 72.15449714660645,
'NeuralNetTorch_BAG_L1/T5': 88.62915325164795,
'WeightedEnsemble_L2': 0.7970461845397949,
'LightGBM_BAG_L2/T1': 7.921231031417847,
'LightGBM_BAG_L2/T2': 7.105218887329102,
'LightGBM_BAG_L2/T3': 7.991039037704468,
'LightGBM_BAG_L2/T4': 7.180105447769165,
'LightGBM_BAG_L2/T5': 7.912818193435669,
'NeuralNetTorch_BAG_L2/T1': 25.693060636520386,
'NeuralNetTorch_BAG_L2/T2': 36.16522741317749,
'NeuralNetTorch_BAG_L2/T3': 35.405948638916016,
'NeuralNetTorch_BAG_L2/T4': 60.85235857963562,
'NeuralNetTorch_BAG_L2/T5': 81.01348757743835,
'WeightedEnsemble_L3': 0.8742287158966064},
'model_pred_times': {'LightGBM_BAG_L1/T1': 0.11072778701782227,
'LightGBM_BAG_L1/T2': 0.12961769104003906,
'LightGBM_BAG_L1/T3': 0.13564085960388184,
'LightGBM_BAG_L1/T4': 0.0945291519165039,
'LightGBM_BAG_L1/T5': 0.11731266975402832,
'NeuralNetTorch_BAG_L1/T1': 0.3858356475830078,
'NeuralNetTorch_BAG_L1/T2': 0.3144950866699219,
'NeuralNetTorch_BAG_L1/T3': 0.33666515350341797,
'NeuralNetTorch_BAG_L1/T4': 0.42746949195861816,
'NeuralNetTorch_BAG_L1/T5': 0.48789024353027344,
'WeightedEnsemble_L2': 0.0008318424224853516,
'LightGBM_BAG_L2/T1': 0.10253620147705078,
'LightGBM_BAG_L2/T2': 0.09523439407348633,
'LightGBM_BAG_L2/T3': 0.14042353630065918,
'LightGBM_BAG_L2/T4': 0.08096933364868164,
'LightGBM_BAG_L2/T5': 0.11015439033508301,
'NeuralNetTorch_BAG_L2/T1': 0.32818102836608887,
'NeuralNetTorch_BAG_L2/T2': 0.3438386917114258,
'NeuralNetTorch_BAG_L2/T3': 0.4233055114746094,
'NeuralNetTorch_BAG_L2/T4': 0.6047353744506836,
'NeuralNetTorch_BAG_L2/T5': 0.7249743938446045,
'WeightedEnsemble_L3': 0.0013158321380615234},
'num_bag_folds': 5,
'max_stack_level': 3,
'model_hyperparams': {'LightGBM_BAG_L1/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -40.706686 5.304729 558.431824
1 LightGBM_BAG_L2/T2 -41.037416 2.635418 302.515243
2 LightGBM_BAG_L2/T1 -41.200410 2.642720 303.331255
3 LightGBM_BAG_L2/T3 -41.346167 2.680607 303.401063
4 LightGBM_BAG_L2/T5 -41.647512 2.650338 303.322842
5 WeightedEnsemble_L2 -42.255533 1.008055 124.509622
6 NeuralNetTorch_BAG_L2/T1 -42.577094 2.868365 321.103085
7 NeuralNetTorch_BAG_L2/T4 -42.582856 3.144919 356.262383
8 NeuralNetTorch_BAG_L2/T2 -42.644782 2.884022 331.575251
9 LightGBM_BAG_L1/T2 -42.847297 0.129618 6.548905
10 LightGBM_BAG_L1/T3 -43.652435 0.135641 6.900732
11 NeuralNetTorch_BAG_L2/T5 -43.880374 3.265158 376.423512
12 LightGBM_BAG_L1/T1 -44.719007 0.110728 7.457866
13 NeuralNetTorch_BAG_L2/T3 -45.610407 2.963489 330.815973
14 LightGBM_BAG_L1/T5 -47.844107 0.117313 7.732576
15 NeuralNetTorch_BAG_L1/T4 -53.814540 0.427469 72.154497
16 NeuralNetTorch_BAG_L1/T2 -56.923024 0.314495 38.108442
17 NeuralNetTorch_BAG_L1/T3 -81.995966 0.336665 35.471487
18 NeuralNetTorch_BAG_L1/T1 -96.490001 0.385836 25.508792
19 LightGBM_BAG_L2/T4 -104.120960 2.621153 302.590129
20 LightGBM_BAG_L1/T4 -124.349788 0.094529 6.897573
21 NeuralNetTorch_BAG_L1/T5 -140.663465 0.487890 88.629153
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.001316 0.874229 3 True
1 0.095234 7.105219 2 True
2 0.102536 7.921231 2 True
3 0.140424 7.991039 2 True
4 0.110154 7.912818 2 True
5 0.000832 0.797046 2 True
6 0.328181 25.693061 2 True
7 0.604735 60.852359 2 True
8 0.343839 36.165227 2 True
9 0.129618 6.548905 1 True
10 0.135641 6.900732 1 True
11 0.724974 81.013488 2 True
12 0.110728 7.457866 1 True
13 0.423306 35.405949 2 True
14 0.117313 7.732576 1 True
15 0.427469 72.154497 1 True
16 0.314495 38.108442 1 True
17 0.336665 35.471487 1 True
18 0.385836 25.508792 1 True
19 0.080969 7.180105 2 True
20 0.094529 6.897573 1 True
21 0.487890 88.629153 1 True
fit_order
0 22
1 13
2 12
3 14
4 16
5 11
6 17
7 20
8 18
9 2
10 3
11 21
12 1
13 19
14 5
15 9
16 7
17 8
18 6
19 15
20 4
21 10 }
Create predictions for train dataset
In [54]:
#predictor_new_hpo
predictor_new_hpo_train = predictor_new_hpo.predict_proba(train)
rmse_new_hpo = mean_squared_error(train['count'], predictor_new_hpo_train, squared=False)
print('The RMSE for predictor new hpo is: ',rmse_new_hpo)
The RMSE for predictor new hpo is: 33.322557743463065
Create predictions for test dataset
In [55]:
#predictor_new_hpo
predictions_new_hpo = predictor_new_hpo.predict_proba(test)
In [56]:
submission_new_hpo = pd.DataFrame({'datetime': submission['datetime'],
'count': predictions_new_hpo})
submission_new_hpo.loc[submission_new_hpo['count'] < 0, 'count'] = 0
In [57]:
# Same submitting predictions
submission_new_hpo['count'] = submission_new_hpo['count'].fillna(0).astype(int)
submission_new_hpo['count'] = submission_new_hpo['count'].astype(np.int64)
submission_new_hpo.to_csv("submission_new_hpo.csv", index=False)
In [58]:
!kaggle competitions submit -c bike-sharing-demand -f submission_new_hpo.csv -m "new features with hyperparameters"
100%|█████████████████████████████████████████| 149k/149k [00:00<00:00, 199kB/s] Successfully submitted to Bike Sharing Demand
In [59]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore --------------------------- ------------------- --------------------------------- -------- ----------- ------------ submission_new_hpo.csv 2022-05-21 12:42:39 new features with hyperparameters complete 0.47659 0.47659 submission_new_features.csv 2022-05-21 12:28:51 new features complete 0.46603 0.46603 submission.csv 2022-05-21 12:08:15 first raw submission complete 1.82081 1.82081 submission_new_hpo.csv 2022-05-19 10:02:34 new features with hyperparameters complete 0.46282 0.46282
New Score of 0.47659¶
Trying different combinations of Hyperparameters - version 2¶
In [79]:
predictor_new_hpo_v2 = TabularPredictor(
label="count",
eval_metric='root_mean_squared_error',
learner_kwargs={'ignored_columns': ["casual", "registered"]}
).fit(
train_data=train,
time_limit=1200,
num_bag_folds=6,
num_bag_sets=1,
num_stack_levels=2,
presets="best_quality",
hyperparameters = {'NN_TORCH': {'num_epochs': 2}, 'GBM': {'num_boost_round': 20}},
hyperparameter_tune_kwargs='auto',
)
No path specified. Models will be saved in: "AutogluonModels/ag-20220521_134143/"
Presets specified: ['best_quality']
Warning: hyperparameter tuning is currently experimental and may cause the process to hang.
Beginning AutoGluon training ... Time limit = 1200s
AutoGluon will save models to "AutogluonModels/ag-20220521_134143/"
AutoGluon Version: 0.4.1
Python Version: 3.7.10
Operating System: Linux
Train Data Rows: 10886
Train Data Columns: 14
Label Column: count
Preprocessing data ...
AutoGluon infers your prediction problem is: 'regression' (because dtype of label-column == int and many unique label-values observed).
Label info (max, min, mean, stddev): (977, 1, 191.57413, 181.14445)
If 'regression' is not the correct problem_type, please manually specify the problem_type parameter during predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression'])
Using Feature Generators to preprocess the data ...
Dropping user-specified ignored columns: ['casual', 'registered']
Fitting AutoMLPipelineFeatureGenerator...
Available Memory: 1967.59 MB
Train Data (Original) Memory Usage: 0.74 MB (0.0% of available memory)
Inferring data type of each feature based on column values. Set feature_metadata_in to manually specify special dtypes of the features.
Stage 1 Generators:
Fitting AsTypeFeatureGenerator...
Note: Converting 3 features to boolean dtype as they only contain 2 unique values.
Stage 2 Generators:
Fitting FillNaFeatureGenerator...
Stage 3 Generators:
Fitting IdentityFeatureGenerator...
Fitting CategoryFeatureGenerator...
Fitting CategoryMemoryMinimizeFeatureGenerator...
Stage 4 Generators:
Fitting DropUniqueFeatureGenerator...
Types of features in original data (raw dtype, special dtypes):
('category', []) : 4 | ['season', 'holiday', 'workingday', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 5 | ['humidity', 'year', 'month', 'day', 'hour']
Types of features in processed data (raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
0.1s = Fit runtime
12 features in original data used to generate 12 features in processed data.
Train Data (Processed) Memory Usage: 0.67 MB (0.0% of available memory)
Data preprocessing and feature engineering runtime = 0.16s ...
AutoGluon will gauge predictive performance using evaluation metric: 'root_mean_squared_error'
To change this, specify the eval_metric parameter of Predictor()
AutoGluon will fit 3 stack levels (L1 to L3) ...
Fitting 2 L1 models ...
Hyperparameter tuning model: LightGBM_BAG_L1 ... Tuning model for up to 39.98s of the 1199.84s of remaining time.
Ran out of time, early stopping on iteration 1. Best iteration is:
[1] valid_set's rmse: 180.547
Stopping HPO to satisfy time limit...
Fitted model: LightGBM_BAG_L1/T1 ...
-96.1594 = Validation score (root_mean_squared_error)
0.25s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T2 ...
-71.2354 = Validation score (root_mean_squared_error)
0.29s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T3 ...
-90.8859 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T4 ...
-168.1893 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T5 ...
-62.8971 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T6 ...
-137.0979 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T7 ...
-61.584 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T8 ...
-61.6155 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T9 ...
-162.6852 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T10 ...
-47.2738 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T11 ...
-152.0219 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T12 ...
-163.3942 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T13 ...
-156.4626 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T14 ...
-134.9397 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T15 ...
-168.3672 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T16 ...
-163.8489 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T17 ...
-168.9279 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T18 ...
-98.1435 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T19 ...
-103.6901 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T20 ...
-102.5023 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T21 ...
-143.9416 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T22 ...
-100.2582 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T23 ...
-97.9426 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T24 ...
-48.9969 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T25 ...
-168.3117 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T26 ...
-149.2756 = Validation score (root_mean_squared_error)
0.25s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T27 ...
-167.9788 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T28 ...
-152.403 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T29 ...
-106.8089 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T30 ...
-59.483 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T31 ...
-148.4568 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T32 ...
-160.2138 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T33 ...
-113.0535 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T34 ...
-67.9513 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T35 ...
-83.579 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T36 ...
-63.8211 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T37 ...
-60.5162 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T38 ...
-101.2134 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T39 ...
-84.9816 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T40 ...
-135.251 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T41 ...
-92.298 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T42 ...
-127.3798 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T43 ...
-119.9997 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T44 ...
-83.7239 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T45 ...
-163.3203 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T46 ...
-57.4981 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T47 ...
-48.4945 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T48 ...
-163.7653 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T49 ...
-159.4538 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T50 ...
-128.7126 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T51 ...
-56.6156 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T52 ...
-168.9781 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T53 ...
-156.6159 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T54 ...
-67.2121 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T55 ...
-150.6513 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T56 ...
-154.0453 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T57 ...
-98.83 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T58 ...
-100.567 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T59 ...
-93.4209 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T60 ...
-54.9384 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T61 ...
-47.8343 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T62 ...
-75.0451 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T63 ...
-125.976 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T64 ...
-89.8658 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T65 ...
-84.805 = Validation score (root_mean_squared_error)
0.49s = Training runtime
0.04s = Validation runtime
Fitted model: LightGBM_BAG_L1/T66 ...
-93.101 = Validation score (root_mean_squared_error)
0.41s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T67 ...
-53.1081 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T68 ...
-142.6263 = Validation score (root_mean_squared_error)
0.31s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T69 ...
-57.8068 = Validation score (root_mean_squared_error)
0.27s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T70 ...
-146.4761 = Validation score (root_mean_squared_error)
0.25s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T71 ...
-45.0568 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T72 ...
-52.3685 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T73 ...
-158.3782 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T74 ...
-158.1351 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T75 ...
-123.3684 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T76 ...
-53.1857 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T77 ...
-162.2208 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T78 ...
-56.8812 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T79 ...
-165.8162 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T80 ...
-164.8504 = Validation score (root_mean_squared_error)
0.27s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L1/T81 ...
-73.609 = Validation score (root_mean_squared_error)
0.31s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L1/T82 ...
-156.9424 = Validation score (root_mean_squared_error)
0.3s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T83 ...
-55.5956 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T84 ...
-146.8305 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T85 ...
-97.4384 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T86 ...
-154.4029 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T87 ...
-113.4807 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T88 ...
-114.6658 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T89 ...
-46.2614 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T90 ...
-49.4682 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T91 ...
-136.4768 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T92 ...
-116.418 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T93 ...
-56.8044 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T94 ...
-48.5038 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T95 ...
-166.6625 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T96 ...
-164.8862 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T97 ...
-171.4948 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T98 ...
-169.8671 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T99 ...
-151.3003 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T100 ...
-50.9384 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T101 ...
-163.6301 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T102 ...
-124.9231 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T103 ...
-137.3387 = Validation score (root_mean_squared_error)
0.24s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T104 ...
-100.4694 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T105 ...
-154.2182 = Validation score (root_mean_squared_error)
0.21s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T106 ...
-53.0285 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T107 ...
-45.9895 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T108 ...
-126.3663 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T109 ...
-46.4378 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T110 ...
-138.44 = Validation score (root_mean_squared_error)
0.2s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T111 ...
-78.7875 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T112 ...
-138.63 = Validation score (root_mean_squared_error)
0.22s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T113 ...
-50.3731 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T114 ...
-51.1664 = Validation score (root_mean_squared_error)
0.23s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L1/T115 ...
-180.5469 = Validation score (root_mean_squared_error)
0.19s = Training runtime
0.01s = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L1 ... Tuning model for up to 39.98s of the 1155.85s of remaining time.
Stopping HPO to satisfy time limit...
Fitted model: NeuralNetTorch_BAG_L1/T1 ...
-138.793 = Validation score (root_mean_squared_error)
0.94s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T2 ...
-105.4212 = Validation score (root_mean_squared_error)
1.64s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T3 ...
-121.3764 = Validation score (root_mean_squared_error)
1.37s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T4 ...
-102.7891 = Validation score (root_mean_squared_error)
1.22s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T5 ...
-124.832 = Validation score (root_mean_squared_error)
2.31s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T6 ...
-96.9827 = Validation score (root_mean_squared_error)
1.5s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T7 ...
-148.9562 = Validation score (root_mean_squared_error)
1.12s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T8 ...
-132.9172 = Validation score (root_mean_squared_error)
0.99s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T9 ...
-127.5594 = Validation score (root_mean_squared_error)
1.2s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T10 ...
-127.8206 = Validation score (root_mean_squared_error)
1.22s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T11 ...
-99.7462 = Validation score (root_mean_squared_error)
1.88s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T12 ...
-105.7916 = Validation score (root_mean_squared_error)
0.86s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T13 ...
-135.9016 = Validation score (root_mean_squared_error)
3.42s = Training runtime
0.06s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T14 ...
-140.0589 = Validation score (root_mean_squared_error)
1.0s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T15 ...
-97.6155 = Validation score (root_mean_squared_error)
2.22s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T16 ...
-109.0504 = Validation score (root_mean_squared_error)
2.86s = Training runtime
0.05s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T17 ...
-129.0148 = Validation score (root_mean_squared_error)
1.15s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L1/T18 ...
-143.8835 = Validation score (root_mean_squared_error)
2.76s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T1 ... Training model for up to 452.46s of the 1119.17s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-95.2464 = Validation score (root_mean_squared_error)
10.0s = Training runtime
0.04s = Validation runtime
Fitting model: LightGBM_BAG_L1/T2 ... Training model for up to 439.53s of the 1106.24s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-70.4303 = Validation score (root_mean_squared_error)
9.12s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T3 ... Training model for up to 427.96s of the 1094.67s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-89.9804 = Validation score (root_mean_squared_error)
9.34s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T4 ... Training model for up to 415.56s of the 1082.27s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-165.429 = Validation score (root_mean_squared_error)
9.58s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T5 ... Training model for up to 402.97s of the 1069.68s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-62.1499 = Validation score (root_mean_squared_error)
8.97s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T6 ... Training model for up to 391.61s of the 1058.32s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-134.9459 = Validation score (root_mean_squared_error)
9.39s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T7 ... Training model for up to 379.79s of the 1046.5s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-61.4308 = Validation score (root_mean_squared_error)
9.29s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T8 ... Training model for up to 367.38s of the 1034.09s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-61.5958 = Validation score (root_mean_squared_error)
9.49s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T9 ... Training model for up to 355.13s of the 1021.84s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-160.2092 = Validation score (root_mean_squared_error)
9.1s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T10 ... Training model for up to 343.71s of the 1010.42s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-46.9856 = Validation score (root_mean_squared_error)
9.39s = Training runtime
0.09s = Validation runtime
Fitting model: LightGBM_BAG_L1/T11 ... Training model for up to 331.98s of the 998.69s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-149.4088 = Validation score (root_mean_squared_error)
10.0s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T12 ... Training model for up to 318.82s of the 985.53s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-160.8696 = Validation score (root_mean_squared_error)
9.38s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T13 ... Training model for up to 306.47s of the 973.18s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-154.0386 = Validation score (root_mean_squared_error)
9.2s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T14 ... Training model for up to 294.88s of the 961.59s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-132.4655 = Validation score (root_mean_squared_error)
9.06s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T15 ... Training model for up to 283.42s of the 950.13s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-165.8398 = Validation score (root_mean_squared_error)
10.26s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T16 ... Training model for up to 269.98s of the 936.69s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-161.0969 = Validation score (root_mean_squared_error)
10.56s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T17 ... Training model for up to 256.37s of the 923.08s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-166.2581 = Validation score (root_mean_squared_error)
9.65s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T18 ... Training model for up to 244.08s of the 910.79s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-98.2333 = Validation score (root_mean_squared_error)
9.48s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T19 ... Training model for up to 232.04s of the 898.75s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-102.7429 = Validation score (root_mean_squared_error)
9.56s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T20 ... Training model for up to 220.01s of the 886.72s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-101.2778 = Validation score (root_mean_squared_error)
10.57s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T21 ... Training model for up to 206.41s of the 873.12s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-141.5044 = Validation score (root_mean_squared_error)
10.07s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T22 ... Training model for up to 193.81s of the 860.52s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-99.9158 = Validation score (root_mean_squared_error)
9.56s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T23 ... Training model for up to 181.73s of the 848.44s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-96.7825 = Validation score (root_mean_squared_error)
10.14s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T24 ... Training model for up to 168.4s of the 835.11s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-48.8118 = Validation score (root_mean_squared_error)
9.29s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L1/T25 ... Training model for up to 156.61s of the 823.32s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-165.7382 = Validation score (root_mean_squared_error)
9.39s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T26 ... Training model for up to 144.03s of the 810.74s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-146.8745 = Validation score (root_mean_squared_error)
10.2s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T27 ... Training model for up to 130.5s of the 797.21s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-165.2447 = Validation score (root_mean_squared_error)
10.12s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T28 ... Training model for up to 117.38s of the 784.09s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-150.1905 = Validation score (root_mean_squared_error)
9.47s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T29 ... Training model for up to 105.54s of the 772.25s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-105.026 = Validation score (root_mean_squared_error)
10.45s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T30 ... Training model for up to 92.07s of the 758.78s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-59.3014 = Validation score (root_mean_squared_error)
9.83s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L1/T31 ... Training model for up to 78.69s of the 745.4s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-146.3214 = Validation score (root_mean_squared_error)
10.55s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L1/T32 ... Training model for up to 65.0s of the 731.71s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-157.5559 = Validation score (root_mean_squared_error)
9.77s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L1/T33 ... Training model for up to 51.93s of the 718.64s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-111.5454 = Validation score (root_mean_squared_error)
9.89s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L1/T34 ... Training model for up to 38.95s of the 705.66s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-67.2108 = Validation score (root_mean_squared_error)
10.79s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T35 ... Training model for up to 25.0s of the 691.71s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-82.7133 = Validation score (root_mean_squared_error)
9.59s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L1/T36 ... Training model for up to 12.89s of the 679.6s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-63.9508 = Validation score (root_mean_squared_error)
9.49s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L1/T37 ... Training model for up to 0.34s of the 667.05s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-152.9645 = Validation score (root_mean_squared_error)
9.95s = Training runtime
0.05s = Validation runtime
Fitting model: WeightedEnsemble_L2 ... Training model for up to 360.0s of the 651.19s of remaining time.
-46.7793 = Validation score (root_mean_squared_error)
0.48s = Training runtime
0.0s = Validation runtime
Fitting 2 L2 models ...
Hyperparameter tuning model: LightGBM_BAG_L2 ... Tuning model for up to 32.52s of the 650.55s of remaining time.
Stopping HPO to satisfy time limit...
Fitted model: LightGBM_BAG_L2/T1 ...
-77.8056 = Validation score (root_mean_squared_error)
0.42s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T2 ...
-61.3433 = Validation score (root_mean_squared_error)
0.51s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T3 ...
-77.7141 = Validation score (root_mean_squared_error)
0.44s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T4 ...
-161.3252 = Validation score (root_mean_squared_error)
0.47s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T5 ...
-50.6378 = Validation score (root_mean_squared_error)
0.43s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T6 ...
-131.3877 = Validation score (root_mean_squared_error)
0.54s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T7 ...
-53.5851 = Validation score (root_mean_squared_error)
0.49s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T8 ...
-51.9891 = Validation score (root_mean_squared_error)
0.42s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T9 ...
-156.7206 = Validation score (root_mean_squared_error)
0.5s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T10 ...
-46.4821 = Validation score (root_mean_squared_error)
0.47s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T11 ...
-141.5367 = Validation score (root_mean_squared_error)
0.47s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T12 ...
-155.3037 = Validation score (root_mean_squared_error)
0.44s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T13 ...
-146.6177 = Validation score (root_mean_squared_error)
0.54s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T14 ...
-112.6792 = Validation score (root_mean_squared_error)
0.36s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L2/T15 ...
-161.1489 = Validation score (root_mean_squared_error)
0.45s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T16 ...
-155.7941 = Validation score (root_mean_squared_error)
0.79s = Training runtime
0.02s = Validation runtime
Fitted model: LightGBM_BAG_L2/T17 ...
-162.5445 = Validation score (root_mean_squared_error)
0.64s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T18 ...
-72.6803 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T19 ...
-86.2001 = Validation score (root_mean_squared_error)
0.39s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T20 ...
-77.9314 = Validation score (root_mean_squared_error)
0.39s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T21 ...
-134.4647 = Validation score (root_mean_squared_error)
0.46s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T22 ...
-71.052 = Validation score (root_mean_squared_error)
0.36s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T23 ...
-68.9161 = Validation score (root_mean_squared_error)
0.36s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T24 ...
-46.1152 = Validation score (root_mean_squared_error)
0.38s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T25 ...
-162.5596 = Validation score (root_mean_squared_error)
0.48s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T26 ...
-137.4156 = Validation score (root_mean_squared_error)
0.47s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T27 ...
-160.8301 = Validation score (root_mean_squared_error)
0.45s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T28 ...
-141.4941 = Validation score (root_mean_squared_error)
0.39s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T29 ...
-87.1743 = Validation score (root_mean_squared_error)
0.46s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T30 ...
-48.811 = Validation score (root_mean_squared_error)
0.4s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T31 ...
-141.5471 = Validation score (root_mean_squared_error)
0.45s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T32 ...
-152.5716 = Validation score (root_mean_squared_error)
0.42s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T33 ...
-88.7714 = Validation score (root_mean_squared_error)
0.38s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T34 ...
-55.7494 = Validation score (root_mean_squared_error)
0.48s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T35 ...
-74.4511 = Validation score (root_mean_squared_error)
0.5s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T36 ...
-49.0472 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T37 ...
-50.4294 = Validation score (root_mean_squared_error)
0.5s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T38 ...
-86.2722 = Validation score (root_mean_squared_error)
0.41s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T39 ...
-70.7915 = Validation score (root_mean_squared_error)
0.48s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T40 ...
-122.1557 = Validation score (root_mean_squared_error)
0.47s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T41 ...
-70.929 = Validation score (root_mean_squared_error)
0.4s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T42 ...
-116.2684 = Validation score (root_mean_squared_error)
0.49s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T43 ...
-88.9744 = Validation score (root_mean_squared_error)
0.35s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T44 ...
-71.7539 = Validation score (root_mean_squared_error)
0.49s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T45 ...
-154.7645 = Validation score (root_mean_squared_error)
0.41s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T46 ...
-50.7888 = Validation score (root_mean_squared_error)
0.46s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T47 ...
-46.8989 = Validation score (root_mean_squared_error)
0.44s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T48 ...
-154.1139 = Validation score (root_mean_squared_error)
0.36s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T49 ...
-152.4578 = Validation score (root_mean_squared_error)
0.44s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T50 ...
-119.415 = Validation score (root_mean_squared_error)
0.44s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T51 ...
-46.6812 = Validation score (root_mean_squared_error)
0.34s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L2/T52 ...
-161.7798 = Validation score (root_mean_squared_error)
0.42s = Training runtime
0.01s = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L2 ... Tuning model for up to 32.52s of the 619.09s of remaining time.
Stopping HPO to satisfy time limit...
Fitted model: NeuralNetTorch_BAG_L2/T1 ...
-52.6743 = Validation score (root_mean_squared_error)
1.28s = Training runtime
0.08s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T2 ...
-50.9213 = Validation score (root_mean_squared_error)
1.73s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T3 ...
-67.2741 = Validation score (root_mean_squared_error)
1.93s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T4 ...
-53.2503 = Validation score (root_mean_squared_error)
1.48s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T5 ...
-62.0811 = Validation score (root_mean_squared_error)
2.44s = Training runtime
0.28s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T6 ...
-52.6096 = Validation score (root_mean_squared_error)
1.77s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T7 ...
-52.95 = Validation score (root_mean_squared_error)
1.22s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T8 ...
-51.4724 = Validation score (root_mean_squared_error)
1.29s = Training runtime
0.08s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T9 ...
-51.1214 = Validation score (root_mean_squared_error)
1.47s = Training runtime
0.08s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T10 ...
-60.8847 = Validation score (root_mean_squared_error)
1.64s = Training runtime
0.08s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T11 ...
-47.6594 = Validation score (root_mean_squared_error)
2.26s = Training runtime
0.09s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T12 ...
-51.9495 = Validation score (root_mean_squared_error)
0.93s = Training runtime
0.03s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L2/T13 ...
-98.2698 = Validation score (root_mean_squared_error)
3.53s = Training runtime
0.1s = Validation runtime
Fitting model: LightGBM_BAG_L2/T1 ... Training model for up to 373.43s of the 590.35s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-77.1487 = Validation score (root_mean_squared_error)
11.52s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T2 ... Training model for up to 359.05s of the 575.96s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-60.1188 = Validation score (root_mean_squared_error)
10.67s = Training runtime
0.08s = Validation runtime
Fitting model: LightGBM_BAG_L2/T3 ... Training model for up to 346.07s of the 562.98s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-77.0813 = Validation score (root_mean_squared_error)
10.85s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T4 ... Training model for up to 332.81s of the 549.72s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-161.4144 = Validation score (root_mean_squared_error)
10.94s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T5 ... Training model for up to 319.09s of the 536.01s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-48.5297 = Validation score (root_mean_squared_error)
10.92s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T6 ... Training model for up to 305.24s of the 522.16s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-131.3732 = Validation score (root_mean_squared_error)
10.88s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T7 ... Training model for up to 291.69s of the 508.6s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-51.7099 = Validation score (root_mean_squared_error)
10.53s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T8 ... Training model for up to 278.58s of the 495.49s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-49.7948 = Validation score (root_mean_squared_error)
10.6s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T9 ... Training model for up to 265.81s of the 482.72s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-156.791 = Validation score (root_mean_squared_error)
10.96s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T10 ... Training model for up to 251.93s of the 468.83s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-43.937 = Validation score (root_mean_squared_error)
11.39s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T11 ... Training model for up to 237.45s of the 454.36s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-141.5751 = Validation score (root_mean_squared_error)
10.84s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T12 ... Training model for up to 223.64s of the 440.55s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-155.3657 = Validation score (root_mean_squared_error)
10.33s = Training runtime
0.09s = Validation runtime
Fitting model: LightGBM_BAG_L2/T13 ... Training model for up to 211.12s of the 428.03s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-146.6166 = Validation score (root_mean_squared_error)
10.76s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T14 ... Training model for up to 197.53s of the 414.44s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-112.6371 = Validation score (root_mean_squared_error)
10.41s = Training runtime
0.11s = Validation runtime
Fitting model: LightGBM_BAG_L2/T15 ... Training model for up to 184.86s of the 401.76s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-161.265 = Validation score (root_mean_squared_error)
10.21s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T16 ... Training model for up to 171.48s of the 388.4s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-155.8769 = Validation score (root_mean_squared_error)
11.17s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T17 ... Training model for up to 157.71s of the 374.62s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-162.6224 = Validation score (root_mean_squared_error)
11.74s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T18 ... Training model for up to 143.13s of the 360.04s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-71.8659 = Validation score (root_mean_squared_error)
10.41s = Training runtime
0.05s = Validation runtime
Fitting model: LightGBM_BAG_L2/T19 ... Training model for up to 129.55s of the 346.46s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-85.8107 = Validation score (root_mean_squared_error)
11.45s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T20 ... Training model for up to 114.77s of the 331.68s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-77.2805 = Validation score (root_mean_squared_error)
10.2s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T21 ... Training model for up to 102.01s of the 318.92s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-134.4673 = Validation score (root_mean_squared_error)
10.5s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T22 ... Training model for up to 89.01s of the 305.92s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-70.1588 = Validation score (root_mean_squared_error)
10.57s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T23 ... Training model for up to 75.86s of the 292.77s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-67.9272 = Validation score (root_mean_squared_error)
10.37s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T24 ... Training model for up to 63.05s of the 279.96s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-43.571 = Validation score (root_mean_squared_error)
10.31s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T25 ... Training model for up to 50.3s of the 267.21s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-162.6221 = Validation score (root_mean_squared_error)
11.15s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L2/T26 ... Training model for up to 36.19s of the 253.08s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-137.4171 = Validation score (root_mean_squared_error)
11.17s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L2/T27 ... Training model for up to 21.91s of the 238.82s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-160.9142 = Validation score (root_mean_squared_error)
10.47s = Training runtime
0.08s = Validation runtime
Fitting model: LightGBM_BAG_L2/T28 ... Training model for up to 8.97s of the 225.88s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-141.6145 = Validation score (root_mean_squared_error)
11.03s = Training runtime
0.07s = Validation runtime
Fitting model: WeightedEnsemble_L3 ... Training model for up to 360.0s of the 209.89s of remaining time.
-43.4603 = Validation score (root_mean_squared_error)
0.35s = Training runtime
0.0s = Validation runtime
Fitting 2 L3 models ...
Hyperparameter tuning model: LightGBM_BAG_L3 ... Tuning model for up to 15.71s of the 209.39s of remaining time.
Stopping HPO to satisfy time limit...
Fitted model: LightGBM_BAG_L3/T1 ...
-76.7346 = Validation score (root_mean_squared_error)
0.35s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T2 ...
-61.0555 = Validation score (root_mean_squared_error)
0.41s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T3 ...
-76.9387 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T4 ...
-160.8138 = Validation score (root_mean_squared_error)
0.39s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T5 ...
-50.0197 = Validation score (root_mean_squared_error)
0.35s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T6 ...
-130.7797 = Validation score (root_mean_squared_error)
0.43s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T7 ...
-52.9297 = Validation score (root_mean_squared_error)
0.45s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T8 ...
-50.989 = Validation score (root_mean_squared_error)
0.35s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T9 ...
-156.1403 = Validation score (root_mean_squared_error)
0.41s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T10 ...
-46.3789 = Validation score (root_mean_squared_error)
0.39s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T11 ...
-140.9668 = Validation score (root_mean_squared_error)
0.38s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T12 ...
-154.6995 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T13 ...
-145.9833 = Validation score (root_mean_squared_error)
0.41s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T14 ...
-111.7448 = Validation score (root_mean_squared_error)
0.31s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T15 ...
-160.5393 = Validation score (root_mean_squared_error)
0.33s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T16 ...
-155.2734 = Validation score (root_mean_squared_error)
0.38s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T17 ...
-162.0223 = Validation score (root_mean_squared_error)
0.4s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T18 ...
-71.4082 = Validation score (root_mean_squared_error)
0.33s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T19 ...
-85.3119 = Validation score (root_mean_squared_error)
0.33s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T20 ...
-76.8438 = Validation score (root_mean_squared_error)
0.33s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T21 ...
-133.8836 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T22 ...
-69.6403 = Validation score (root_mean_squared_error)
0.31s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T23 ...
-67.5269 = Validation score (root_mean_squared_error)
0.31s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T24 ...
-45.7577 = Validation score (root_mean_squared_error)
0.33s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T25 ...
-161.9758 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T26 ...
-136.779 = Validation score (root_mean_squared_error)
0.39s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T27 ...
-160.2776 = Validation score (root_mean_squared_error)
0.37s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T28 ...
-140.8378 = Validation score (root_mean_squared_error)
0.34s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T29 ...
-86.5646 = Validation score (root_mean_squared_error)
0.36s = Training runtime
0.01s = Validation runtime
Fitted model: LightGBM_BAG_L3/T30 ...
-48.1206 = Validation score (root_mean_squared_error)
0.34s = Training runtime
0.01s = Validation runtime
Hyperparameter tuning model: NeuralNetTorch_BAG_L3 ... Tuning model for up to 15.71s of the 193.81s of remaining time.
Stopping HPO to satisfy time limit...
Fitted model: NeuralNetTorch_BAG_L3/T1 ...
-48.5864 = Validation score (root_mean_squared_error)
1.23s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T2 ...
-50.2672 = Validation score (root_mean_squared_error)
1.7s = Training runtime
0.04s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T3 ...
-59.9232 = Validation score (root_mean_squared_error)
1.58s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T4 ...
-51.119 = Validation score (root_mean_squared_error)
1.46s = Training runtime
0.07s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T5 ...
-52.6298 = Validation score (root_mean_squared_error)
2.43s = Training runtime
0.08s = Validation runtime
Fitted model: NeuralNetTorch_BAG_L3/T6 ...
-54.5518 = Validation score (root_mean_squared_error)
1.92s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L3/T1 ... Training model for up to 180.96s of the 180.91s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-76.6953 = Validation score (root_mean_squared_error)
10.42s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T2 ... Training model for up to 167.16s of the 167.11s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-60.3636 = Validation score (root_mean_squared_error)
10.75s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T3 ... Training model for up to 153.84s of the 153.79s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-76.8015 = Validation score (root_mean_squared_error)
10.53s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L3/T4 ... Training model for up to 140.23s of the 140.18s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-161.3706 = Validation score (root_mean_squared_error)
10.47s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T5 ... Training model for up to 126.44s of the 126.39s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-48.6524 = Validation score (root_mean_squared_error)
10.82s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L3/T6 ... Training model for up to 112.34s of the 112.29s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-131.2738 = Validation score (root_mean_squared_error)
14.98s = Training runtime
0.08s = Validation runtime
Fitting model: LightGBM_BAG_L3/T7 ... Training model for up to 94.28s of the 94.23s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-51.7651 = Validation score (root_mean_squared_error)
10.51s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L3/T8 ... Training model for up to 81.4s of the 81.35s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-49.6566 = Validation score (root_mean_squared_error)
10.47s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T9 ... Training model for up to 67.92s of the 67.87s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-156.7114 = Validation score (root_mean_squared_error)
11.09s = Training runtime
0.07s = Validation runtime
Fitting model: LightGBM_BAG_L3/T10 ... Training model for up to 53.96s of the 53.91s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-44.3452 = Validation score (root_mean_squared_error)
11.27s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T11 ... Training model for up to 39.33s of the 39.28s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-141.4654 = Validation score (root_mean_squared_error)
10.66s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T12 ... Training model for up to 25.4s of the 25.35s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-155.2612 = Validation score (root_mean_squared_error)
10.28s = Training runtime
0.06s = Validation runtime
Fitting model: LightGBM_BAG_L3/T13 ... Training model for up to 12.61s of the 12.56s of remaining time.
Fitting 5 child models (S1F2 - S1F6) | Fitting with ParallelLocalFoldFittingStrategy
-146.5451 = Validation score (root_mean_squared_error)
10.35s = Training runtime
0.07s = Validation runtime
Fitting model: WeightedEnsemble_L4 ... Training model for up to 360.0s of the -1.63s of remaining time.
-44.3452 = Validation score (root_mean_squared_error)
0.35s = Training runtime
0.0s = Validation runtime
AutoGluon training complete, total runtime = 1202.27s ... Best model: "WeightedEnsemble_L3"
TabularPredictor saved. To load, use: predictor = TabularPredictor.load("AutogluonModels/ag-20220521_134143/")
In [80]:
predictor_new_hpo_v2.fit_summary()
*** Summary of fit() ***
Estimated performance of each model:
model score_val pred_time_val fit_time pred_time_val_marginal fit_time_marginal stack_level can_infer fit_order
0 WeightedEnsemble_L3 -43.460319 2.306577 381.974152 0.000876 0.347950 3 True 200
1 LightGBM_BAG_L2/T24 -43.570961 2.235956 370.239415 0.072493 10.308636 2 True 158
2 LightGBM_BAG_L2/T10 -43.937039 2.233209 371.317567 0.069745 11.386788 2 True 144
3 LightGBM_BAG_L3/T10 -44.345180 4.141941 673.566589 0.064293 11.269752 3 True 210
4 WeightedEnsemble_L4 -44.345180 4.142688 673.915158 0.000747 0.348568 4 True 237
5 LightGBM_BAG_L1/T71 -45.056774 0.008519 0.235917 0.008519 0.235917 1 True 71
6 LightGBM_BAG_L3/T24 -45.757680 4.084984 662.625002 0.007337 0.328164 3 True 224
7 LightGBM_BAG_L1/T107 -45.989459 0.007218 0.229095 0.007218 0.229095 1 True 107
8 LightGBM_BAG_L1/T89 -46.261418 0.007159 0.224034 0.007159 0.224034 1 True 89
9 LightGBM_BAG_L1/T109 -46.437759 0.007054 0.221164 0.007054 0.221164 1 True 109
10 LightGBM_BAG_L2/T51 -46.681152 2.170503 360.266608 0.007039 0.335829 2 True 185
11 WeightedEnsemble_L2 -46.779286 0.163266 19.163258 0.001148 0.475723 2 True 134
12 LightGBM_BAG_L2/T47 -46.898943 2.171569 360.366563 0.008106 0.435784 2 True 181
13 LightGBM_BAG_L1/T10 -46.985623 0.089329 9.394629 0.089329 9.394629 1 True 10
14 NeuralNetTorch_BAG_L2/T11 -47.659355 2.255722 362.188172 0.092258 2.257393 2 True 197
15 LightGBM_BAG_L1/T61 -47.834260 0.007396 0.232562 0.007396 0.232562 1 True 61
16 LightGBM_BAG_L3/T30 -48.120608 4.085141 662.634193 0.007494 0.337355 3 True 230
17 LightGBM_BAG_L1/T47 -48.494512 0.007655 0.225170 0.007655 0.225170 1 True 47
18 LightGBM_BAG_L1/T94 -48.503779 0.006870 0.220363 0.006870 0.220363 1 True 94
19 LightGBM_BAG_L2/T5 -48.529699 2.227863 370.850576 0.064399 10.919797 2 True 139
20 NeuralNetTorch_BAG_L3/T1 -48.586389 4.147132 663.528044 0.069485 1.231207 3 True 231
21 LightGBM_BAG_L3/T5 -48.652445 4.146297 673.116226 0.068650 10.819388 3 True 205
22 LightGBM_BAG_L2/T30 -48.811046 2.171432 360.326760 0.007968 0.395981 2 True 164
23 LightGBM_BAG_L1/T24 -48.811782 0.072790 9.292907 0.072790 9.292907 1 True 24
24 LightGBM_BAG_L2/T36 -49.047246 2.174662 360.296565 0.011198 0.365786 2 True 170
25 LightGBM_BAG_L1/T90 -49.468162 0.006792 0.206065 0.006792 0.206065 1 True 90
26 LightGBM_BAG_L3/T8 -49.656635 4.133121 672.771686 0.055474 10.474848 3 True 208
27 LightGBM_BAG_L2/T8 -49.794838 2.231861 370.528136 0.068398 10.597358 2 True 142
28 NeuralNetTorch_BAG_L3/T2 -50.267170 4.115871 663.997388 0.038224 1.700550 3 True 232
29 LightGBM_BAG_L1/T113 -50.373094 0.007121 0.230512 0.007121 0.230512 1 True 113
30 LightGBM_BAG_L2/T37 -50.429360 2.172501 360.431727 0.009037 0.500948 2 True 171
31 LightGBM_BAG_L2/T46 -50.788790 2.172910 360.387473 0.009446 0.456694 2 True 180
32 NeuralNetTorch_BAG_L2/T2 -50.921303 2.198969 361.662731 0.035506 1.731952 2 True 188
33 LightGBM_BAG_L1/T100 -50.938412 0.007288 0.228707 0.007288 0.228707 1 True 100
34 NeuralNetTorch_BAG_L3/T4 -51.118959 4.143447 663.759203 0.065799 1.462365 3 True 234
35 NeuralNetTorch_BAG_L2/T9 -51.121433 2.244301 361.399956 0.080838 1.469177 2 True 195
36 LightGBM_BAG_L1/T114 -51.166393 0.012602 0.228076 0.012602 0.228076 1 True 114
37 NeuralNetTorch_BAG_L2/T8 -51.472392 2.238962 361.224025 0.075498 1.293246 2 True 194
38 LightGBM_BAG_L2/T7 -51.709883 2.233213 370.459575 0.069749 10.528796 2 True 141
39 LightGBM_BAG_L3/T7 -51.765128 4.149523 672.806423 0.071875 10.509585 3 True 207
40 NeuralNetTorch_BAG_L2/T12 -51.949523 2.190379 360.860381 0.026915 0.929602 2 True 198
41 LightGBM_BAG_L1/T72 -52.368527 0.009573 0.212278 0.009573 0.212278 1 True 72
42 NeuralNetTorch_BAG_L2/T6 -52.609616 2.232281 361.699302 0.068818 1.768523 2 True 192
43 NeuralNetTorch_BAG_L3/T5 -52.629777 4.153352 664.726630 0.075705 2.429793 3 True 235
44 NeuralNetTorch_BAG_L2/T1 -52.674276 2.239338 361.214180 0.075874 1.283401 2 True 187
45 NeuralNetTorch_BAG_L2/T7 -52.950038 2.238297 361.153965 0.074833 1.223186 2 True 193
46 LightGBM_BAG_L1/T106 -53.028500 0.008013 0.222290 0.008013 0.222290 1 True 106
47 LightGBM_BAG_L1/T67 -53.108142 0.008053 0.244969 0.008053 0.244969 1 True 67
48 LightGBM_BAG_L1/T76 -53.185713 0.007575 0.220364 0.007575 0.220364 1 True 76
49 NeuralNetTorch_BAG_L2/T4 -53.250292 2.230874 361.407783 0.067410 1.477004 2 True 190
50 NeuralNetTorch_BAG_L3/T6 -54.551765 4.149120 664.217582 0.071473 1.920745 3 True 236
51 LightGBM_BAG_L1/T60 -54.938450 0.007218 0.210299 0.007218 0.210299 1 True 60
52 LightGBM_BAG_L1/T83 -55.595622 0.006056 0.207053 0.006056 0.207053 1 True 83
53 LightGBM_BAG_L2/T34 -55.749416 2.171851 360.406150 0.008388 0.475371 2 True 168
54 LightGBM_BAG_L1/T51 -56.615568 0.006451 0.204403 0.006451 0.204403 1 True 51
55 LightGBM_BAG_L1/T93 -56.804420 0.006592 0.222801 0.006592 0.222801 1 True 93
56 LightGBM_BAG_L1/T78 -56.881182 0.007219 0.226611 0.007219 0.226611 1 True 78
57 LightGBM_BAG_L1/T46 -57.498094 0.007246 0.231942 0.007246 0.231942 1 True 46
58 LightGBM_BAG_L1/T69 -57.806844 0.007620 0.269982 0.007620 0.269982 1 True 69
59 LightGBM_BAG_L1/T30 -59.301369 0.065185 9.833703 0.065185 9.833703 1 True 30
60 NeuralNetTorch_BAG_L3/T3 -59.923184 4.149328 663.878519 0.071681 1.581681 3 True 233
61 LightGBM_BAG_L2/T2 -60.118779 2.241281 370.604797 0.077817 10.674018 2 True 136
62 LightGBM_BAG_L3/T2 -60.363647 4.139754 673.042545 0.062107 10.745707 3 True 202
63 NeuralNetTorch_BAG_L2/T10 -60.884651 2.239229 361.569122 0.075765 1.638343 2 True 196
64 LightGBM_BAG_L1/T7 -61.430823 0.058785 9.292935 0.058785 9.292935 1 True 7
65 LightGBM_BAG_L1/T8 -61.595818 0.059454 9.493112 0.059454 9.493112 1 True 8
66 NeuralNetTorch_BAG_L2/T5 -62.081090 2.439783 362.374147 0.276320 2.443368 2 True 191
67 LightGBM_BAG_L1/T5 -62.149900 0.058731 8.969120 0.058731 8.969120 1 True 5
68 LightGBM_BAG_L1/T36 -63.950820 0.065624 9.486712 0.065624 9.486712 1 True 36
69 LightGBM_BAG_L1/T34 -67.210837 0.063193 10.786390 0.063193 10.786390 1 True 34
70 LightGBM_BAG_L1/T54 -67.212089 0.006682 0.218171 0.006682 0.218171 1 True 54
71 NeuralNetTorch_BAG_L2/T3 -67.274105 2.237578 361.859015 0.074114 1.928236 2 True 189
72 LightGBM_BAG_L3/T23 -67.526948 4.084053 662.609949 0.006406 0.313112 3 True 223
73 LightGBM_BAG_L2/T23 -67.927218 2.222276 370.304873 0.058812 10.374094 2 True 157
74 LightGBM_BAG_L3/T22 -69.640327 4.084576 662.607048 0.006929 0.310210 3 True 222
75 LightGBM_BAG_L2/T22 -70.158818 2.221691 370.503316 0.058227 10.572537 2 True 156
76 LightGBM_BAG_L1/T2 -70.430295 0.062552 9.124408 0.062552 9.124408 1 True 2
77 LightGBM_BAG_L2/T39 -70.791530 2.172120 360.408942 0.008656 0.478163 2 True 173
78 LightGBM_BAG_L2/T41 -70.928971 2.170843 360.327948 0.007380 0.397169 2 True 175
79 LightGBM_BAG_L3/T18 -71.408163 4.083952 662.626612 0.006305 0.329774 3 True 218
80 LightGBM_BAG_L2/T44 -71.753892 2.171257 360.419116 0.007794 0.488338 2 True 178
81 LightGBM_BAG_L2/T18 -71.865931 2.217102 370.344557 0.053638 10.413778 2 True 152
82 LightGBM_BAG_L1/T81 -73.609044 0.015997 0.307613 0.015997 0.307613 1 True 81
83 LightGBM_BAG_L2/T35 -74.451067 2.171357 360.434605 0.007894 0.503826 2 True 169
84 LightGBM_BAG_L1/T62 -75.045088 0.007167 0.230537 0.007167 0.230537 1 True 62
85 LightGBM_BAG_L3/T1 -76.695278 4.132939 672.718109 0.055292 10.421272 3 True 201
86 LightGBM_BAG_L3/T3 -76.801488 4.147330 672.829674 0.069683 10.532836 3 True 203
87 LightGBM_BAG_L3/T20 -76.843757 4.086789 662.622042 0.009142 0.325205 3 True 220
88 LightGBM_BAG_L2/T3 -77.081298 2.232345 370.779327 0.068881 10.848548 2 True 137
89 LightGBM_BAG_L2/T1 -77.148745 2.223167 371.448302 0.059703 11.517523 2 True 135
90 LightGBM_BAG_L2/T20 -77.280486 2.229053 370.134429 0.065590 10.203650 2 True 154
91 LightGBM_BAG_L1/T111 -78.787486 0.007246 0.230157 0.007246 0.230157 1 True 111
92 LightGBM_BAG_L1/T35 -82.713341 0.056009 9.594652 0.056009 9.594652 1 True 35
93 LightGBM_BAG_L1/T44 -83.723886 0.008006 0.233032 0.008006 0.233032 1 True 44
94 LightGBM_BAG_L1/T65 -84.805025 0.038642 0.487516 0.038642 0.487516 1 True 65
95 LightGBM_BAG_L1/T39 -84.981571 0.006960 0.222264 0.006960 0.222264 1 True 39
96 LightGBM_BAG_L3/T19 -85.311924 4.083980 662.631718 0.006332 0.334881 3 True 219
97 LightGBM_BAG_L2/T19 -85.810719 2.226915 371.381443 0.063451 11.450664 2 True 153
98 LightGBM_BAG_L2/T38 -86.272226 2.170388 360.343070 0.006925 0.412291 2 True 172
99 LightGBM_BAG_L3/T29 -86.564560 4.086278 662.656795 0.008631 0.359957 3 True 229
100 LightGBM_BAG_L2/T29 -87.174273 2.170688 360.390271 0.007225 0.459492 2 True 163
101 LightGBM_BAG_L2/T33 -88.771423 2.174581 360.306565 0.011117 0.375786 2 True 167
102 LightGBM_BAG_L2/T43 -88.974427 2.171088 360.281029 0.007625 0.350250 2 True 177
103 LightGBM_BAG_L1/T64 -89.865774 0.006758 0.242683 0.006758 0.242683 1 True 64
104 LightGBM_BAG_L1/T3 -89.980400 0.055822 9.343952 0.055822 9.343952 1 True 3
105 LightGBM_BAG_L1/T41 -92.297995 0.007127 0.216170 0.007127 0.216170 1 True 41
106 LightGBM_BAG_L1/T66 -93.100964 0.013462 0.412814 0.013462 0.412814 1 True 66
107 LightGBM_BAG_L1/T59 -93.420888 0.007372 0.225325 0.007372 0.225325 1 True 59
108 LightGBM_BAG_L1/T1 -95.246392 0.044828 10.002664 0.044828 10.002664 1 True 1
109 LightGBM_BAG_L1/T23 -96.782527 0.050156 10.137167 0.050156 10.137167 1 True 23
110 NeuralNetTorch_BAG_L1/T6 -96.982748 0.034503 1.504171 0.034503 1.504171 1 True 121
111 LightGBM_BAG_L1/T85 -97.438360 0.012075 0.235161 0.012075 0.235161 1 True 85
112 NeuralNetTorch_BAG_L1/T15 -97.615502 0.042824 2.215915 0.042824 2.215915 1 True 130
113 LightGBM_BAG_L1/T18 -98.233263 0.050075 9.483726 0.050075 9.483726 1 True 18
114 NeuralNetTorch_BAG_L2/T13 -98.269755 2.261712 363.456774 0.098249 3.525995 2 True 199
115 LightGBM_BAG_L1/T57 -98.830000 0.006837 0.217807 0.006837 0.217807 1 True 57
116 NeuralNetTorch_BAG_L1/T11 -99.746162 0.041610 1.879334 0.041610 1.879334 1 True 126
117 LightGBM_BAG_L1/T22 -99.915800 0.049458 9.555924 0.049458 9.555924 1 True 22
118 LightGBM_BAG_L1/T104 -100.469381 0.005826 0.210403 0.005826 0.210403 1 True 104
119 LightGBM_BAG_L1/T58 -100.567002 0.006989 0.222640 0.006989 0.222640 1 True 58
120 LightGBM_BAG_L1/T38 -101.213383 0.006991 0.215770 0.006991 0.215770 1 True 38
121 LightGBM_BAG_L1/T20 -101.277845 0.055207 10.566669 0.055207 10.566669 1 True 20
122 LightGBM_BAG_L1/T19 -102.742899 0.057548 9.555108 0.057548 9.555108 1 True 19
123 NeuralNetTorch_BAG_L1/T4 -102.789106 0.030295 1.222421 0.030295 1.222421 1 True 119
124 LightGBM_BAG_L1/T29 -105.025974 0.055869 10.453753 0.055869 10.453753 1 True 29
125 NeuralNetTorch_BAG_L1/T2 -105.421228 0.032702 1.639454 0.032702 1.639454 1 True 117
126 NeuralNetTorch_BAG_L1/T12 -105.791639 0.028919 0.860202 0.028919 0.860202 1 True 127
127 NeuralNetTorch_BAG_L1/T16 -109.050418 0.047965 2.855595 0.047965 2.855595 1 True 131
128 LightGBM_BAG_L1/T33 -111.545426 0.051340 9.889948 0.051340 9.889948 1 True 33
129 LightGBM_BAG_L3/T14 -111.744829 4.084137 662.607153 0.006490 0.310315 3 True 214
130 LightGBM_BAG_L2/T14 -112.637071 2.270706 370.338713 0.107242 10.407934 2 True 148
131 LightGBM_BAG_L1/T87 -113.480674 0.006609 0.222663 0.006609 0.222663 1 True 87
132 LightGBM_BAG_L1/T88 -114.665779 0.006663 0.223322 0.006663 0.223322 1 True 88
133 LightGBM_BAG_L2/T42 -116.268409 2.171167 360.416349 0.007703 0.485570 2 True 176
134 LightGBM_BAG_L1/T92 -116.418050 0.007560 0.240565 0.007560 0.240565 1 True 92
135 LightGBM_BAG_L2/T50 -119.415014 2.170832 360.369703 0.007369 0.438924 2 True 184
136 LightGBM_BAG_L1/T43 -119.999745 0.005975 0.202012 0.005975 0.202012 1 True 43
137 NeuralNetTorch_BAG_L1/T3 -121.376442 0.034544 1.368851 0.034544 1.368851 1 True 118
138 LightGBM_BAG_L2/T40 -122.155652 2.171247 360.404101 0.007783 0.473322 2 True 174
139 LightGBM_BAG_L1/T75 -123.368369 0.007195 0.239283 0.007195 0.239283 1 True 75
140 NeuralNetTorch_BAG_L1/T5 -124.831996 0.035971 2.305893 0.035971 2.305893 1 True 120
141 LightGBM_BAG_L1/T102 -124.923122 0.006633 0.226857 0.006633 0.226857 1 True 102
142 LightGBM_BAG_L1/T63 -125.976026 0.006454 0.233655 0.006454 0.233655 1 True 63
143 LightGBM_BAG_L1/T108 -126.366307 0.006500 0.215496 0.006500 0.215496 1 True 108
144 LightGBM_BAG_L1/T42 -127.379753 0.007006 0.232998 0.007006 0.232998 1 True 42
145 NeuralNetTorch_BAG_L1/T9 -127.559371 0.035177 1.199696 0.035177 1.199696 1 True 124
146 NeuralNetTorch_BAG_L1/T10 -127.820613 0.037171 1.223811 0.037171 1.223811 1 True 125
147 LightGBM_BAG_L1/T50 -128.712555 0.006351 0.220665 0.006351 0.220665 1 True 50
148 NeuralNetTorch_BAG_L1/T17 -129.014777 0.037956 1.146091 0.037956 1.146091 1 True 132
149 LightGBM_BAG_L3/T6 -131.273761 4.153743 677.279923 0.076096 14.983086 3 True 206
150 LightGBM_BAG_L2/T6 -131.373153 2.229738 370.811903 0.066275 10.881124 2 True 140
151 LightGBM_BAG_L1/T14 -132.465502 0.052489 9.055940 0.052489 9.055940 1 True 14
152 NeuralNetTorch_BAG_L1/T8 -132.917200 0.026577 0.992594 0.026577 0.992594 1 True 123
153 LightGBM_BAG_L3/T21 -133.883575 4.084583 662.666624 0.006936 0.369787 3 True 221
154 LightGBM_BAG_L2/T21 -134.467303 2.225352 370.430907 0.061889 10.500129 2 True 155
155 LightGBM_BAG_L1/T6 -134.945865 0.056398 9.385328 0.056398 9.385328 1 True 6
156 LightGBM_BAG_L1/T40 -135.250951 0.006876 0.240702 0.006876 0.240702 1 True 40
157 NeuralNetTorch_BAG_L1/T13 -135.901553 0.056993 3.418016 0.056993 3.418016 1 True 128
158 LightGBM_BAG_L1/T91 -136.476828 0.006179 0.210755 0.006179 0.210755 1 True 91
159 LightGBM_BAG_L3/T26 -136.779011 4.084606 662.686901 0.006958 0.390064 3 True 226
160 LightGBM_BAG_L1/T103 -137.338684 0.006614 0.235855 0.006614 0.235855 1 True 103
161 LightGBM_BAG_L2/T26 -137.417098 2.233890 371.101029 0.070426 11.170250 2 True 160
162 LightGBM_BAG_L1/T110 -138.439960 0.006432 0.202126 0.006432 0.202126 1 True 110
163 LightGBM_BAG_L1/T112 -138.629973 0.007081 0.215525 0.007081 0.215525 1 True 112
164 NeuralNetTorch_BAG_L1/T1 -138.792950 0.031776 0.941310 0.031776 0.941310 1 True 116
165 NeuralNetTorch_BAG_L1/T14 -140.058903 0.025222 0.996935 0.025222 0.996935 1 True 129
166 LightGBM_BAG_L3/T28 -140.837839 4.084552 662.632007 0.006905 0.335170 3 True 228
167 LightGBM_BAG_L3/T11 -141.465351 4.138753 672.961007 0.061106 10.664170 3 True 211
168 LightGBM_BAG_L1/T21 -141.504419 0.054090 10.074440 0.054090 10.074440 1 True 21
169 LightGBM_BAG_L2/T31 -141.547077 2.171096 360.377096 0.007633 0.446317 2 True 165
170 LightGBM_BAG_L2/T11 -141.575098 2.234422 370.771731 0.070959 10.840952 2 True 145
171 LightGBM_BAG_L2/T28 -141.614454 2.228727 370.963150 0.065263 11.032371 2 True 162
172 LightGBM_BAG_L1/T68 -142.626323 0.008566 0.307189 0.008566 0.307189 1 True 68
173 NeuralNetTorch_BAG_L1/T18 -143.883466 0.045452 2.756621 0.045452 2.756621 1 True 133
174 LightGBM_BAG_L1/T31 -146.321359 0.065132 10.546894 0.065132 10.546894 1 True 31
175 LightGBM_BAG_L1/T70 -146.476142 0.007033 0.251943 0.007033 0.251943 1 True 70
176 LightGBM_BAG_L3/T13 -146.545146 4.149428 672.643589 0.071781 10.346751 3 True 213
177 LightGBM_BAG_L2/T13 -146.616564 2.236465 370.694702 0.073001 10.763923 2 True 147
178 LightGBM_BAG_L1/T84 -146.830452 0.006954 0.232655 0.006954 0.232655 1 True 84
179 LightGBM_BAG_L1/T26 -146.874510 0.058922 10.196182 0.058922 10.196182 1 True 26
180 NeuralNetTorch_BAG_L1/T7 -148.956213 0.025508 1.118420 0.025508 1.118420 1 True 122
181 LightGBM_BAG_L1/T11 -149.408781 0.057441 9.997264 0.057441 9.997264 1 True 11
182 LightGBM_BAG_L1/T28 -150.190487 0.053971 9.465527 0.053971 9.465527 1 True 28
183 LightGBM_BAG_L1/T55 -150.651261 0.008810 0.202813 0.008810 0.202813 1 True 55
184 LightGBM_BAG_L1/T99 -151.300289 0.007069 0.224231 0.007069 0.224231 1 True 99
185 LightGBM_BAG_L2/T49 -152.457810 2.171077 360.371604 0.007613 0.440825 2 True 183
186 LightGBM_BAG_L2/T32 -152.571604 2.170635 360.352623 0.007172 0.421844 2 True 166
187 LightGBM_BAG_L1/T37 -152.964503 0.047522 9.946213 0.047522 9.946213 1 True 37
188 LightGBM_BAG_L1/T13 -154.038604 0.058502 9.197129 0.058502 9.197129 1 True 13
189 LightGBM_BAG_L1/T56 -154.045318 0.006758 0.231108 0.006758 0.231108 1 True 56
190 LightGBM_BAG_L2/T48 -154.113926 2.171930 360.288404 0.008466 0.357625 2 True 182
191 LightGBM_BAG_L1/T105 -154.218162 0.008882 0.214792 0.008882 0.214792 1 True 105
192 LightGBM_BAG_L1/T86 -154.402871 0.006605 0.233217 0.006605 0.233217 1 True 86
193 LightGBM_BAG_L2/T45 -154.764457 2.172047 360.342645 0.008584 0.411866 2 True 179
194 LightGBM_BAG_L3/T12 -155.261240 4.141278 672.574313 0.063630 10.277476 3 True 212
195 LightGBM_BAG_L3/T16 -155.273360 4.084779 662.671921 0.007131 0.375083 3 True 216
196 LightGBM_BAG_L2/T12 -155.365703 2.257148 370.259859 0.093684 10.329080 2 True 146
197 LightGBM_BAG_L2/T16 -155.876853 2.237416 371.102615 0.073953 11.171836 2 True 150
198 LightGBM_BAG_L1/T53 -156.615881 0.007701 0.218027 0.007701 0.218027 1 True 53
199 LightGBM_BAG_L3/T9 -156.711372 4.145249 673.390725 0.067601 11.093887 3 True 209
200 LightGBM_BAG_L2/T9 -156.791000 2.227316 370.887554 0.063853 10.956775 2 True 143
201 LightGBM_BAG_L1/T82 -156.942402 0.006402 0.295640 0.006402 0.295640 1 True 82
202 LightGBM_BAG_L1/T32 -157.555922 0.066901 9.766403 0.066901 9.766403 1 True 32
203 LightGBM_BAG_L1/T74 -158.135111 0.007795 0.225794 0.007795 0.225794 1 True 74
204 LightGBM_BAG_L1/T73 -158.378232 0.006136 0.211785 0.006136 0.211785 1 True 73
205 LightGBM_BAG_L1/T49 -159.453823 0.008553 0.222695 0.008553 0.222695 1 True 49
206 LightGBM_BAG_L1/T9 -160.209171 0.053844 9.100260 0.053844 9.100260 1 True 9
207 LightGBM_BAG_L3/T27 -160.277604 4.085126 662.662751 0.007478 0.365914 3 True 227
208 LightGBM_BAG_L3/T15 -160.539259 4.084344 662.624089 0.006697 0.327252 3 True 215
209 LightGBM_BAG_L1/T12 -160.869643 0.063613 9.382915 0.063613 9.382915 1 True 12
210 LightGBM_BAG_L2/T27 -160.914226 2.240627 370.398039 0.077163 10.467260 2 True 161
211 LightGBM_BAG_L1/T16 -161.096862 0.064341 10.561400 0.064341 10.561400 1 True 16
212 LightGBM_BAG_L2/T15 -161.265009 2.221536 370.144864 0.058072 10.214085 2 True 149
213 LightGBM_BAG_L3/T4 -161.370641 4.138125 672.768961 0.060478 10.472124 3 True 204
214 LightGBM_BAG_L2/T4 -161.414420 2.222565 370.875491 0.059102 10.944712 2 True 138
215 LightGBM_BAG_L2/T52 -161.779784 2.171278 360.352561 0.007815 0.421782 2 True 186
216 LightGBM_BAG_L3/T25 -161.975775 4.084645 662.669852 0.006998 0.373014 3 True 225
217 LightGBM_BAG_L3/T17 -162.022256 4.084596 662.695612 0.006949 0.398774 3 True 217
218 LightGBM_BAG_L1/T77 -162.220783 0.006598 0.219930 0.006598 0.219930 1 True 77
219 LightGBM_BAG_L2/T25 -162.622067 2.222416 371.083233 0.058953 11.152454 2 True 159
220 LightGBM_BAG_L2/T17 -162.622413 2.226909 371.667764 0.063446 11.736985 2 True 151
221 LightGBM_BAG_L1/T45 -163.320324 0.006821 0.216607 0.006821 0.216607 1 True 45
222 LightGBM_BAG_L1/T101 -163.630065 0.007937 0.223276 0.007937 0.223276 1 True 101
223 LightGBM_BAG_L1/T48 -163.765306 0.006855 0.215949 0.006855 0.215949 1 True 48
224 LightGBM_BAG_L1/T80 -164.850379 0.017365 0.269749 0.017365 0.269749 1 True 80
225 LightGBM_BAG_L1/T96 -164.886155 0.006708 0.209600 0.006708 0.209600 1 True 96
226 LightGBM_BAG_L1/T27 -165.244724 0.058286 10.115333 0.058286 10.115333 1 True 27
227 LightGBM_BAG_L1/T4 -165.428965 0.054300 9.584543 0.054300 9.584543 1 True 4
228 LightGBM_BAG_L1/T25 -165.738197 0.055545 9.386972 0.055545 9.386972 1 True 25
229 LightGBM_BAG_L1/T79 -165.816201 0.009422 0.230140 0.009422 0.230140 1 True 79
230 LightGBM_BAG_L1/T15 -165.839770 0.059432 10.260672 0.059432 10.260672 1 True 15
231 LightGBM_BAG_L1/T17 -166.258117 0.060778 9.649885 0.060778 9.649885 1 True 17
232 LightGBM_BAG_L1/T95 -166.662518 0.007148 0.225159 0.007148 0.225159 1 True 95
233 LightGBM_BAG_L1/T52 -168.978088 0.007133 0.225075 0.007133 0.225075 1 True 52
234 LightGBM_BAG_L1/T98 -169.867051 0.007392 0.201504 0.007392 0.201504 1 True 98
235 LightGBM_BAG_L1/T97 -171.494836 0.006243 0.206049 0.006243 0.206049 1 True 97
236 LightGBM_BAG_L1/T115 -180.546931 0.006215 0.189158 0.006215 0.189158 1 True 115
Number of models trained: 237
Types of models trained:
{'WeightedEnsembleModel', 'StackerEnsembleModel_LGB', 'StackerEnsembleModel_TabularNeuralNetTorch'}
Bagging used: True (with 6 folds)
Multi-layer stack-ensembling used: True (with 4 levels)
Feature Metadata (Processed):
(raw dtype, special dtypes):
('category', []) : 2 | ['season', 'weather']
('float', []) : 3 | ['temp', 'atemp', 'windspeed']
('int', []) : 4 | ['humidity', 'month', 'day', 'hour']
('int', ['bool']) : 3 | ['holiday', 'workingday', 'year']
Plot summary of models saved to file: AutogluonModels/ag-20220521_134143/SummaryOfModels.html
*** End of fit() summary ***
Out[80]:
{'model_types': {'LightGBM_BAG_L1/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T3': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T4': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T5': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T6': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T7': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T8': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T9': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T10': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T11': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T12': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T13': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T14': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T15': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T16': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T17': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T18': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T19': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T20': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T21': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T22': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T23': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T24': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T25': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T26': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T27': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T28': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T29': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T30': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T31': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T32': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T33': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T34': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T35': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T36': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T37': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T38': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T39': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T40': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T41': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T42': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T43': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T44': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T45': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T46': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T47': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T48': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T49': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T50': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T51': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T52': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T53': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T54': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T55': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T56': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T57': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T58': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T59': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T60': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T61': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T62': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T63': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T64': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T65': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T66': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T67': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T68': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T69': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T70': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T71': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T72': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T73': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T74': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T75': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T76': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T77': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T78': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T79': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T80': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T81': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T82': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T83': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T84': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T85': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T86': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T87': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T88': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T89': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T90': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T91': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T92': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T93': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T94': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T95': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T96': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T97': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T98': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T99': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T100': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T101': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T102': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T103': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T104': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T105': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T106': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T107': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T108': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T109': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T110': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T111': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T112': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T113': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T114': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L1/T115': 'StackerEnsembleModel_LGB',
'NeuralNetTorch_BAG_L1/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T6': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T7': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T8': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T9': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T10': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T11': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T12': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T13': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T14': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T15': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T16': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T17': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L1/T18': 'StackerEnsembleModel_TabularNeuralNetTorch',
'WeightedEnsemble_L2': 'WeightedEnsembleModel',
'LightGBM_BAG_L2/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T3': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T4': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T5': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T6': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T7': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T8': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T9': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T10': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T11': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T12': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T13': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T14': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T15': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T16': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T17': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T18': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T19': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T20': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T21': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T22': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T23': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T24': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T25': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T26': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T27': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T28': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T29': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T30': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T31': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T32': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T33': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T34': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T35': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T36': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T37': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T38': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T39': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T40': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T41': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T42': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T43': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T44': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T45': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T46': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T47': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T48': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T49': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T50': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T51': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L2/T52': 'StackerEnsembleModel_LGB',
'NeuralNetTorch_BAG_L2/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T6': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T7': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T8': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T9': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T10': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T11': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T12': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L2/T13': 'StackerEnsembleModel_TabularNeuralNetTorch',
'WeightedEnsemble_L3': 'WeightedEnsembleModel',
'LightGBM_BAG_L3/T1': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T2': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T3': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T4': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T5': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T6': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T7': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T8': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T9': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T10': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T11': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T12': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T13': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T14': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T15': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T16': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T17': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T18': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T19': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T20': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T21': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T22': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T23': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T24': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T25': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T26': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T27': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T28': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T29': 'StackerEnsembleModel_LGB',
'LightGBM_BAG_L3/T30': 'StackerEnsembleModel_LGB',
'NeuralNetTorch_BAG_L3/T1': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L3/T2': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L3/T3': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L3/T4': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L3/T5': 'StackerEnsembleModel_TabularNeuralNetTorch',
'NeuralNetTorch_BAG_L3/T6': 'StackerEnsembleModel_TabularNeuralNetTorch',
'WeightedEnsemble_L4': 'WeightedEnsembleModel'},
'model_performance': {'LightGBM_BAG_L1/T1': -95.24639164559781,
'LightGBM_BAG_L1/T2': -70.43029457979837,
'LightGBM_BAG_L1/T3': -89.98039976556805,
'LightGBM_BAG_L1/T4': -165.4289650418463,
'LightGBM_BAG_L1/T5': -62.14989960908567,
'LightGBM_BAG_L1/T6': -134.94586520147033,
'LightGBM_BAG_L1/T7': -61.43082287932244,
'LightGBM_BAG_L1/T8': -61.59581820619491,
'LightGBM_BAG_L1/T9': -160.20917072309442,
'LightGBM_BAG_L1/T10': -46.9856225471341,
'LightGBM_BAG_L1/T11': -149.40878138097298,
'LightGBM_BAG_L1/T12': -160.8696430394228,
'LightGBM_BAG_L1/T13': -154.0386043723983,
'LightGBM_BAG_L1/T14': -132.4655023288866,
'LightGBM_BAG_L1/T15': -165.8397699719795,
'LightGBM_BAG_L1/T16': -161.09686164048298,
'LightGBM_BAG_L1/T17': -166.2581169684571,
'LightGBM_BAG_L1/T18': -98.23326332950049,
'LightGBM_BAG_L1/T19': -102.74289872874851,
'LightGBM_BAG_L1/T20': -101.27784472431108,
'LightGBM_BAG_L1/T21': -141.50441866062513,
'LightGBM_BAG_L1/T22': -99.91580049713026,
'LightGBM_BAG_L1/T23': -96.782526933767,
'LightGBM_BAG_L1/T24': -48.81178173808668,
'LightGBM_BAG_L1/T25': -165.73819684652204,
'LightGBM_BAG_L1/T26': -146.87451027496323,
'LightGBM_BAG_L1/T27': -165.24472378003523,
'LightGBM_BAG_L1/T28': -150.1904867526079,
'LightGBM_BAG_L1/T29': -105.02597407360135,
'LightGBM_BAG_L1/T30': -59.30136946794056,
'LightGBM_BAG_L1/T31': -146.32135936203926,
'LightGBM_BAG_L1/T32': -157.55592226067927,
'LightGBM_BAG_L1/T33': -111.54542573676088,
'LightGBM_BAG_L1/T34': -67.21083671487322,
'LightGBM_BAG_L1/T35': -82.71334070935556,
'LightGBM_BAG_L1/T36': -63.95082015638741,
'LightGBM_BAG_L1/T37': -152.964503489081,
'LightGBM_BAG_L1/T38': -101.21338320908495,
'LightGBM_BAG_L1/T39': -84.98157102825932,
'LightGBM_BAG_L1/T40': -135.2509514475015,
'LightGBM_BAG_L1/T41': -92.29799521482289,
'LightGBM_BAG_L1/T42': -127.37975329766945,
'LightGBM_BAG_L1/T43': -119.99974521147973,
'LightGBM_BAG_L1/T44': -83.72388625714288,
'LightGBM_BAG_L1/T45': -163.32032355618733,
'LightGBM_BAG_L1/T46': -57.49809414956977,
'LightGBM_BAG_L1/T47': -48.49451171986661,
'LightGBM_BAG_L1/T48': -163.76530597989898,
'LightGBM_BAG_L1/T49': -159.45382307965372,
'LightGBM_BAG_L1/T50': -128.71255538821234,
'LightGBM_BAG_L1/T51': -56.615567759363756,
'LightGBM_BAG_L1/T52': -168.97808762258396,
'LightGBM_BAG_L1/T53': -156.61588112932367,
'LightGBM_BAG_L1/T54': -67.21208914900417,
'LightGBM_BAG_L1/T55': -150.6512614160556,
'LightGBM_BAG_L1/T56': -154.0453184654329,
'LightGBM_BAG_L1/T57': -98.82999986244965,
'LightGBM_BAG_L1/T58': -100.56700229444995,
'LightGBM_BAG_L1/T59': -93.42088791087528,
'LightGBM_BAG_L1/T60': -54.93844950877705,
'LightGBM_BAG_L1/T61': -47.8342598007438,
'LightGBM_BAG_L1/T62': -75.04508785029742,
'LightGBM_BAG_L1/T63': -125.97602550162962,
'LightGBM_BAG_L1/T64': -89.86577359914915,
'LightGBM_BAG_L1/T65': -84.80502456784234,
'LightGBM_BAG_L1/T66': -93.10096416601286,
'LightGBM_BAG_L1/T67': -53.10814225956848,
'LightGBM_BAG_L1/T68': -142.626322689254,
'LightGBM_BAG_L1/T69': -57.806843896862304,
'LightGBM_BAG_L1/T70': -146.47614183655418,
'LightGBM_BAG_L1/T71': -45.056774342604,
'LightGBM_BAG_L1/T72': -52.3685265689472,
'LightGBM_BAG_L1/T73': -158.37823179945408,
'LightGBM_BAG_L1/T74': -158.13511074616008,
'LightGBM_BAG_L1/T75': -123.3683692203776,
'LightGBM_BAG_L1/T76': -53.18571314452455,
'LightGBM_BAG_L1/T77': -162.22078301969637,
'LightGBM_BAG_L1/T78': -56.88118234973596,
'LightGBM_BAG_L1/T79': -165.81620058357328,
'LightGBM_BAG_L1/T80': -164.85037888119328,
'LightGBM_BAG_L1/T81': -73.6090444350346,
'LightGBM_BAG_L1/T82': -156.94240191957397,
'LightGBM_BAG_L1/T83': -55.595622403587825,
'LightGBM_BAG_L1/T84': -146.83045166745313,
'LightGBM_BAG_L1/T85': -97.4383602196798,
'LightGBM_BAG_L1/T86': -154.40287115460052,
'LightGBM_BAG_L1/T87': -113.48067418370216,
'LightGBM_BAG_L1/T88': -114.66577916739924,
'LightGBM_BAG_L1/T89': -46.26141836245038,
'LightGBM_BAG_L1/T90': -49.46816230332918,
'LightGBM_BAG_L1/T91': -136.47682848555982,
'LightGBM_BAG_L1/T92': -116.41804973423352,
'LightGBM_BAG_L1/T93': -56.80441986734995,
'LightGBM_BAG_L1/T94': -48.50377851532774,
'LightGBM_BAG_L1/T95': -166.66251812150614,
'LightGBM_BAG_L1/T96': -164.88615515887656,
'LightGBM_BAG_L1/T97': -171.49483594639938,
'LightGBM_BAG_L1/T98': -169.86705058656818,
'LightGBM_BAG_L1/T99': -151.30028881687844,
'LightGBM_BAG_L1/T100': -50.93841165301794,
'LightGBM_BAG_L1/T101': -163.63006530141763,
'LightGBM_BAG_L1/T102': -124.92312212400682,
'LightGBM_BAG_L1/T103': -137.3386843303289,
'LightGBM_BAG_L1/T104': -100.46938070079096,
'LightGBM_BAG_L1/T105': -154.2181617756975,
'LightGBM_BAG_L1/T106': -53.02850026381649,
'LightGBM_BAG_L1/T107': -45.9894585580412,
'LightGBM_BAG_L1/T108': -126.36630744696683,
'LightGBM_BAG_L1/T109': -46.43775853220347,
'LightGBM_BAG_L1/T110': -138.43996035484594,
'LightGBM_BAG_L1/T111': -78.7874863886287,
'LightGBM_BAG_L1/T112': -138.62997280215376,
'LightGBM_BAG_L1/T113': -50.37309404546691,
'LightGBM_BAG_L1/T114': -51.16639286146918,
'LightGBM_BAG_L1/T115': -180.5469309330303,
'NeuralNetTorch_BAG_L1/T1': -138.79295027526143,
'NeuralNetTorch_BAG_L1/T2': -105.42122784621523,
'NeuralNetTorch_BAG_L1/T3': -121.37644216000962,
'NeuralNetTorch_BAG_L1/T4': -102.78910610336368,
'NeuralNetTorch_BAG_L1/T5': -124.83199619479261,
'NeuralNetTorch_BAG_L1/T6': -96.9827482169343,
'NeuralNetTorch_BAG_L1/T7': -148.9562126758656,
'NeuralNetTorch_BAG_L1/T8': -132.9172001969884,
'NeuralNetTorch_BAG_L1/T9': -127.55937130584459,
'NeuralNetTorch_BAG_L1/T10': -127.82061346927256,
'NeuralNetTorch_BAG_L1/T11': -99.74616246637306,
'NeuralNetTorch_BAG_L1/T12': -105.79163892386858,
'NeuralNetTorch_BAG_L1/T13': -135.9015526539169,
'NeuralNetTorch_BAG_L1/T14': -140.0589031949081,
'NeuralNetTorch_BAG_L1/T15': -97.6155020257566,
'NeuralNetTorch_BAG_L1/T16': -109.05041776392018,
'NeuralNetTorch_BAG_L1/T17': -129.0147771763527,
'NeuralNetTorch_BAG_L1/T18': -143.88346597908685,
'WeightedEnsemble_L2': -46.779286301222996,
'LightGBM_BAG_L2/T1': -77.14874466928583,
'LightGBM_BAG_L2/T2': -60.11877898625628,
'LightGBM_BAG_L2/T3': -77.08129804733285,
'LightGBM_BAG_L2/T4': -161.41441990765844,
'LightGBM_BAG_L2/T5': -48.52969904011934,
'LightGBM_BAG_L2/T6': -131.37315263182788,
'LightGBM_BAG_L2/T7': -51.709883217749145,
'LightGBM_BAG_L2/T8': -49.79483815388218,
'LightGBM_BAG_L2/T9': -156.79099995282576,
'LightGBM_BAG_L2/T10': -43.93703886087322,
'LightGBM_BAG_L2/T11': -141.57509800949657,
'LightGBM_BAG_L2/T12': -155.36570273557263,
'LightGBM_BAG_L2/T13': -146.6165637385049,
'LightGBM_BAG_L2/T14': -112.63707051801374,
'LightGBM_BAG_L2/T15': -161.2650085125938,
'LightGBM_BAG_L2/T16': -155.87685254432824,
'LightGBM_BAG_L2/T17': -162.62241250526787,
'LightGBM_BAG_L2/T18': -71.86593112531361,
'LightGBM_BAG_L2/T19': -85.81071939841397,
'LightGBM_BAG_L2/T20': -77.28048599041333,
'LightGBM_BAG_L2/T21': -134.4673028564128,
'LightGBM_BAG_L2/T22': -70.15881828725007,
'LightGBM_BAG_L2/T23': -67.92721831361952,
'LightGBM_BAG_L2/T24': -43.57096096168988,
'LightGBM_BAG_L2/T25': -162.62206684887906,
'LightGBM_BAG_L2/T26': -137.4170980060873,
'LightGBM_BAG_L2/T27': -160.91422570686024,
'LightGBM_BAG_L2/T28': -141.61445373488476,
'LightGBM_BAG_L2/T29': -87.17427349216257,
'LightGBM_BAG_L2/T30': -48.81104568687166,
'LightGBM_BAG_L2/T31': -141.54707736112474,
'LightGBM_BAG_L2/T32': -152.5716044129156,
'LightGBM_BAG_L2/T33': -88.77142277326983,
'LightGBM_BAG_L2/T34': -55.749415727507035,
'LightGBM_BAG_L2/T35': -74.45106679994736,
'LightGBM_BAG_L2/T36': -49.047245533855175,
'LightGBM_BAG_L2/T37': -50.42936044068636,
'LightGBM_BAG_L2/T38': -86.2722259935454,
'LightGBM_BAG_L2/T39': -70.79153046093575,
'LightGBM_BAG_L2/T40': -122.15565194437455,
'LightGBM_BAG_L2/T41': -70.92897124704922,
'LightGBM_BAG_L2/T42': -116.26840944458378,
'LightGBM_BAG_L2/T43': -88.97442687485974,
'LightGBM_BAG_L2/T44': -71.75389161928561,
'LightGBM_BAG_L2/T45': -154.7644565990468,
'LightGBM_BAG_L2/T46': -50.78878951892069,
'LightGBM_BAG_L2/T47': -46.89894348888589,
'LightGBM_BAG_L2/T48': -154.1139259581854,
'LightGBM_BAG_L2/T49': -152.4578100263843,
'LightGBM_BAG_L2/T50': -119.41501439177497,
'LightGBM_BAG_L2/T51': -46.68115243160784,
'LightGBM_BAG_L2/T52': -161.77978386743854,
'NeuralNetTorch_BAG_L2/T1': -52.6742758870963,
'NeuralNetTorch_BAG_L2/T2': -50.92130314076267,
'NeuralNetTorch_BAG_L2/T3': -67.27410483933936,
'NeuralNetTorch_BAG_L2/T4': -53.25029244266176,
'NeuralNetTorch_BAG_L2/T5': -62.08109018839035,
'NeuralNetTorch_BAG_L2/T6': -52.60961619644969,
'NeuralNetTorch_BAG_L2/T7': -52.9500383715647,
'NeuralNetTorch_BAG_L2/T8': -51.47239244542595,
'NeuralNetTorch_BAG_L2/T9': -51.12143278692236,
'NeuralNetTorch_BAG_L2/T10': -60.884651060021945,
'NeuralNetTorch_BAG_L2/T11': -47.65935488999549,
'NeuralNetTorch_BAG_L2/T12': -51.94952302764032,
'NeuralNetTorch_BAG_L2/T13': -98.26975527171821,
'WeightedEnsemble_L3': -43.46031947621305,
'LightGBM_BAG_L3/T1': -76.69527796121601,
'LightGBM_BAG_L3/T2': -60.36364733782896,
'LightGBM_BAG_L3/T3': -76.80148772241041,
'LightGBM_BAG_L3/T4': -161.37064127439612,
'LightGBM_BAG_L3/T5': -48.65244531024017,
'LightGBM_BAG_L3/T6': -131.2737606093407,
'LightGBM_BAG_L3/T7': -51.765127695720444,
'LightGBM_BAG_L3/T8': -49.65663526838185,
'LightGBM_BAG_L3/T9': -156.7113720097058,
'LightGBM_BAG_L3/T10': -44.345179837557225,
'LightGBM_BAG_L3/T11': -141.46535113246236,
'LightGBM_BAG_L3/T12': -155.26123995998728,
'LightGBM_BAG_L3/T13': -146.54514588050827,
'LightGBM_BAG_L3/T14': -111.74482884765649,
'LightGBM_BAG_L3/T15': -160.53925896752193,
'LightGBM_BAG_L3/T16': -155.27335951898624,
'LightGBM_BAG_L3/T17': -162.02225552024152,
'LightGBM_BAG_L3/T18': -71.40816313640647,
'LightGBM_BAG_L3/T19': -85.31192368561926,
'LightGBM_BAG_L3/T20': -76.84375696466381,
'LightGBM_BAG_L3/T21': -133.88357530976188,
'LightGBM_BAG_L3/T22': -69.64032723601979,
'LightGBM_BAG_L3/T23': -67.5269483516493,
'LightGBM_BAG_L3/T24': -45.75767965426102,
'LightGBM_BAG_L3/T25': -161.97577535636094,
'LightGBM_BAG_L3/T26': -136.77901114344402,
'LightGBM_BAG_L3/T27': -160.27760438219826,
'LightGBM_BAG_L3/T28': -140.8378390205732,
'LightGBM_BAG_L3/T29': -86.56456014475087,
'LightGBM_BAG_L3/T30': -48.120607886366784,
'NeuralNetTorch_BAG_L3/T1': -48.586389034691415,
'NeuralNetTorch_BAG_L3/T2': -50.26716955215357,
'NeuralNetTorch_BAG_L3/T3': -59.923183703565556,
'NeuralNetTorch_BAG_L3/T4': -51.11895920431928,
'NeuralNetTorch_BAG_L3/T5': -52.629777343342866,
'NeuralNetTorch_BAG_L3/T6': -54.55176483698387,
'WeightedEnsemble_L4': -44.345179837557225},
'model_best': 'WeightedEnsemble_L3',
'model_paths': {'LightGBM_BAG_L1/T1': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T1/',
'LightGBM_BAG_L1/T2': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T2/',
'LightGBM_BAG_L1/T3': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T3/',
'LightGBM_BAG_L1/T4': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T4/',
'LightGBM_BAG_L1/T5': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T5/',
'LightGBM_BAG_L1/T6': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T6/',
'LightGBM_BAG_L1/T7': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T7/',
'LightGBM_BAG_L1/T8': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T8/',
'LightGBM_BAG_L1/T9': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T9/',
'LightGBM_BAG_L1/T10': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T10/',
'LightGBM_BAG_L1/T11': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T11/',
'LightGBM_BAG_L1/T12': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T12/',
'LightGBM_BAG_L1/T13': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T13/',
'LightGBM_BAG_L1/T14': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T14/',
'LightGBM_BAG_L1/T15': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T15/',
'LightGBM_BAG_L1/T16': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T16/',
'LightGBM_BAG_L1/T17': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T17/',
'LightGBM_BAG_L1/T18': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T18/',
'LightGBM_BAG_L1/T19': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T19/',
'LightGBM_BAG_L1/T20': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T20/',
'LightGBM_BAG_L1/T21': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T21/',
'LightGBM_BAG_L1/T22': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T22/',
'LightGBM_BAG_L1/T23': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T23/',
'LightGBM_BAG_L1/T24': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T24/',
'LightGBM_BAG_L1/T25': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T25/',
'LightGBM_BAG_L1/T26': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T26/',
'LightGBM_BAG_L1/T27': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T27/',
'LightGBM_BAG_L1/T28': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T28/',
'LightGBM_BAG_L1/T29': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T29/',
'LightGBM_BAG_L1/T30': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T30/',
'LightGBM_BAG_L1/T31': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T31/',
'LightGBM_BAG_L1/T32': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T32/',
'LightGBM_BAG_L1/T33': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T33/',
'LightGBM_BAG_L1/T34': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T34/',
'LightGBM_BAG_L1/T35': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T35/',
'LightGBM_BAG_L1/T36': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T36/',
'LightGBM_BAG_L1/T37': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T37/',
'LightGBM_BAG_L1/T38': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T38/',
'LightGBM_BAG_L1/T39': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T39/',
'LightGBM_BAG_L1/T40': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T40/',
'LightGBM_BAG_L1/T41': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T41/',
'LightGBM_BAG_L1/T42': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T42/',
'LightGBM_BAG_L1/T43': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T43/',
'LightGBM_BAG_L1/T44': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T44/',
'LightGBM_BAG_L1/T45': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T45/',
'LightGBM_BAG_L1/T46': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T46/',
'LightGBM_BAG_L1/T47': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T47/',
'LightGBM_BAG_L1/T48': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T48/',
'LightGBM_BAG_L1/T49': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T49/',
'LightGBM_BAG_L1/T50': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T50/',
'LightGBM_BAG_L1/T51': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T51/',
'LightGBM_BAG_L1/T52': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T52/',
'LightGBM_BAG_L1/T53': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T53/',
'LightGBM_BAG_L1/T54': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T54/',
'LightGBM_BAG_L1/T55': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T55/',
'LightGBM_BAG_L1/T56': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T56/',
'LightGBM_BAG_L1/T57': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T57/',
'LightGBM_BAG_L1/T58': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T58/',
'LightGBM_BAG_L1/T59': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T59/',
'LightGBM_BAG_L1/T60': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T60/',
'LightGBM_BAG_L1/T61': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T61/',
'LightGBM_BAG_L1/T62': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T62/',
'LightGBM_BAG_L1/T63': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T63/',
'LightGBM_BAG_L1/T64': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T64/',
'LightGBM_BAG_L1/T65': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T65/',
'LightGBM_BAG_L1/T66': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T66/',
'LightGBM_BAG_L1/T67': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T67/',
'LightGBM_BAG_L1/T68': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T68/',
'LightGBM_BAG_L1/T69': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T69/',
'LightGBM_BAG_L1/T70': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T70/',
'LightGBM_BAG_L1/T71': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T71/',
'LightGBM_BAG_L1/T72': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T72/',
'LightGBM_BAG_L1/T73': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T73/',
'LightGBM_BAG_L1/T74': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T74/',
'LightGBM_BAG_L1/T75': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T75/',
'LightGBM_BAG_L1/T76': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T76/',
'LightGBM_BAG_L1/T77': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T77/',
'LightGBM_BAG_L1/T78': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T78/',
'LightGBM_BAG_L1/T79': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T79/',
'LightGBM_BAG_L1/T80': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T80/',
'LightGBM_BAG_L1/T81': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T81/',
'LightGBM_BAG_L1/T82': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T82/',
'LightGBM_BAG_L1/T83': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T83/',
'LightGBM_BAG_L1/T84': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T84/',
'LightGBM_BAG_L1/T85': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T85/',
'LightGBM_BAG_L1/T86': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T86/',
'LightGBM_BAG_L1/T87': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T87/',
'LightGBM_BAG_L1/T88': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T88/',
'LightGBM_BAG_L1/T89': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T89/',
'LightGBM_BAG_L1/T90': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T90/',
'LightGBM_BAG_L1/T91': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T91/',
'LightGBM_BAG_L1/T92': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T92/',
'LightGBM_BAG_L1/T93': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T93/',
'LightGBM_BAG_L1/T94': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T94/',
'LightGBM_BAG_L1/T95': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T95/',
'LightGBM_BAG_L1/T96': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T96/',
'LightGBM_BAG_L1/T97': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T97/',
'LightGBM_BAG_L1/T98': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T98/',
'LightGBM_BAG_L1/T99': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T99/',
'LightGBM_BAG_L1/T100': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T100/',
'LightGBM_BAG_L1/T101': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T101/',
'LightGBM_BAG_L1/T102': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T102/',
'LightGBM_BAG_L1/T103': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T103/',
'LightGBM_BAG_L1/T104': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T104/',
'LightGBM_BAG_L1/T105': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T105/',
'LightGBM_BAG_L1/T106': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T106/',
'LightGBM_BAG_L1/T107': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T107/',
'LightGBM_BAG_L1/T108': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T108/',
'LightGBM_BAG_L1/T109': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T109/',
'LightGBM_BAG_L1/T110': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T110/',
'LightGBM_BAG_L1/T111': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T111/',
'LightGBM_BAG_L1/T112': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T112/',
'LightGBM_BAG_L1/T113': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T113/',
'LightGBM_BAG_L1/T114': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T114/',
'LightGBM_BAG_L1/T115': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L1/T115/',
'NeuralNetTorch_BAG_L1/T1': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T1/',
'NeuralNetTorch_BAG_L1/T2': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T2/',
'NeuralNetTorch_BAG_L1/T3': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T3/',
'NeuralNetTorch_BAG_L1/T4': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T4/',
'NeuralNetTorch_BAG_L1/T5': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T5/',
'NeuralNetTorch_BAG_L1/T6': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T6/',
'NeuralNetTorch_BAG_L1/T7': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T7/',
'NeuralNetTorch_BAG_L1/T8': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T8/',
'NeuralNetTorch_BAG_L1/T9': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T9/',
'NeuralNetTorch_BAG_L1/T10': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T10/',
'NeuralNetTorch_BAG_L1/T11': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T11/',
'NeuralNetTorch_BAG_L1/T12': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T12/',
'NeuralNetTorch_BAG_L1/T13': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T13/',
'NeuralNetTorch_BAG_L1/T14': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T14/',
'NeuralNetTorch_BAG_L1/T15': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T15/',
'NeuralNetTorch_BAG_L1/T16': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T16/',
'NeuralNetTorch_BAG_L1/T17': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T17/',
'NeuralNetTorch_BAG_L1/T18': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L1/T18/',
'WeightedEnsemble_L2': 'AutogluonModels/ag-20220521_134143/models/WeightedEnsemble_L2/',
'LightGBM_BAG_L2/T1': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T1/',
'LightGBM_BAG_L2/T2': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T2/',
'LightGBM_BAG_L2/T3': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T3/',
'LightGBM_BAG_L2/T4': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T4/',
'LightGBM_BAG_L2/T5': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T5/',
'LightGBM_BAG_L2/T6': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T6/',
'LightGBM_BAG_L2/T7': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T7/',
'LightGBM_BAG_L2/T8': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T8/',
'LightGBM_BAG_L2/T9': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T9/',
'LightGBM_BAG_L2/T10': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T10/',
'LightGBM_BAG_L2/T11': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T11/',
'LightGBM_BAG_L2/T12': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T12/',
'LightGBM_BAG_L2/T13': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T13/',
'LightGBM_BAG_L2/T14': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T14/',
'LightGBM_BAG_L2/T15': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T15/',
'LightGBM_BAG_L2/T16': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T16/',
'LightGBM_BAG_L2/T17': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T17/',
'LightGBM_BAG_L2/T18': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T18/',
'LightGBM_BAG_L2/T19': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T19/',
'LightGBM_BAG_L2/T20': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T20/',
'LightGBM_BAG_L2/T21': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T21/',
'LightGBM_BAG_L2/T22': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T22/',
'LightGBM_BAG_L2/T23': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T23/',
'LightGBM_BAG_L2/T24': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T24/',
'LightGBM_BAG_L2/T25': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T25/',
'LightGBM_BAG_L2/T26': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T26/',
'LightGBM_BAG_L2/T27': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T27/',
'LightGBM_BAG_L2/T28': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T28/',
'LightGBM_BAG_L2/T29': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T29/',
'LightGBM_BAG_L2/T30': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T30/',
'LightGBM_BAG_L2/T31': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T31/',
'LightGBM_BAG_L2/T32': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T32/',
'LightGBM_BAG_L2/T33': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T33/',
'LightGBM_BAG_L2/T34': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T34/',
'LightGBM_BAG_L2/T35': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T35/',
'LightGBM_BAG_L2/T36': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T36/',
'LightGBM_BAG_L2/T37': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T37/',
'LightGBM_BAG_L2/T38': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T38/',
'LightGBM_BAG_L2/T39': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T39/',
'LightGBM_BAG_L2/T40': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T40/',
'LightGBM_BAG_L2/T41': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T41/',
'LightGBM_BAG_L2/T42': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T42/',
'LightGBM_BAG_L2/T43': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T43/',
'LightGBM_BAG_L2/T44': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T44/',
'LightGBM_BAG_L2/T45': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T45/',
'LightGBM_BAG_L2/T46': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T46/',
'LightGBM_BAG_L2/T47': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T47/',
'LightGBM_BAG_L2/T48': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T48/',
'LightGBM_BAG_L2/T49': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T49/',
'LightGBM_BAG_L2/T50': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T50/',
'LightGBM_BAG_L2/T51': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T51/',
'LightGBM_BAG_L2/T52': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L2/T52/',
'NeuralNetTorch_BAG_L2/T1': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T1/',
'NeuralNetTorch_BAG_L2/T2': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T2/',
'NeuralNetTorch_BAG_L2/T3': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T3/',
'NeuralNetTorch_BAG_L2/T4': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T4/',
'NeuralNetTorch_BAG_L2/T5': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T5/',
'NeuralNetTorch_BAG_L2/T6': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T6/',
'NeuralNetTorch_BAG_L2/T7': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T7/',
'NeuralNetTorch_BAG_L2/T8': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T8/',
'NeuralNetTorch_BAG_L2/T9': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T9/',
'NeuralNetTorch_BAG_L2/T10': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T10/',
'NeuralNetTorch_BAG_L2/T11': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T11/',
'NeuralNetTorch_BAG_L2/T12': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T12/',
'NeuralNetTorch_BAG_L2/T13': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L2/T13/',
'WeightedEnsemble_L3': 'AutogluonModels/ag-20220521_134143/models/WeightedEnsemble_L3/',
'LightGBM_BAG_L3/T1': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T1/',
'LightGBM_BAG_L3/T2': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T2/',
'LightGBM_BAG_L3/T3': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T3/',
'LightGBM_BAG_L3/T4': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T4/',
'LightGBM_BAG_L3/T5': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T5/',
'LightGBM_BAG_L3/T6': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T6/',
'LightGBM_BAG_L3/T7': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T7/',
'LightGBM_BAG_L3/T8': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T8/',
'LightGBM_BAG_L3/T9': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T9/',
'LightGBM_BAG_L3/T10': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T10/',
'LightGBM_BAG_L3/T11': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T11/',
'LightGBM_BAG_L3/T12': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T12/',
'LightGBM_BAG_L3/T13': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T13/',
'LightGBM_BAG_L3/T14': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T14/',
'LightGBM_BAG_L3/T15': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T15/',
'LightGBM_BAG_L3/T16': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T16/',
'LightGBM_BAG_L3/T17': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T17/',
'LightGBM_BAG_L3/T18': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T18/',
'LightGBM_BAG_L3/T19': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T19/',
'LightGBM_BAG_L3/T20': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T20/',
'LightGBM_BAG_L3/T21': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T21/',
'LightGBM_BAG_L3/T22': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T22/',
'LightGBM_BAG_L3/T23': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T23/',
'LightGBM_BAG_L3/T24': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T24/',
'LightGBM_BAG_L3/T25': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T25/',
'LightGBM_BAG_L3/T26': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T26/',
'LightGBM_BAG_L3/T27': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T27/',
'LightGBM_BAG_L3/T28': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T28/',
'LightGBM_BAG_L3/T29': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T29/',
'LightGBM_BAG_L3/T30': 'AutogluonModels/ag-20220521_134143/models/LightGBM_BAG_L3/T30/',
'NeuralNetTorch_BAG_L3/T1': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T1/',
'NeuralNetTorch_BAG_L3/T2': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T2/',
'NeuralNetTorch_BAG_L3/T3': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T3/',
'NeuralNetTorch_BAG_L3/T4': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T4/',
'NeuralNetTorch_BAG_L3/T5': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T5/',
'NeuralNetTorch_BAG_L3/T6': 'AutogluonModels/ag-20220521_134143/models/NeuralNetTorch_BAG_L3/T6/',
'WeightedEnsemble_L4': 'AutogluonModels/ag-20220521_134143/models/WeightedEnsemble_L4/'},
'model_fit_times': {'LightGBM_BAG_L1/T1': 10.002663850784302,
'LightGBM_BAG_L1/T2': 9.124408483505249,
'LightGBM_BAG_L1/T3': 9.34395170211792,
'LightGBM_BAG_L1/T4': 9.584542512893677,
'LightGBM_BAG_L1/T5': 8.969119787216187,
'LightGBM_BAG_L1/T6': 9.385327816009521,
'LightGBM_BAG_L1/T7': 9.292935132980347,
'LightGBM_BAG_L1/T8': 9.493111610412598,
'LightGBM_BAG_L1/T9': 9.100260019302368,
'LightGBM_BAG_L1/T10': 9.394628763198853,
'LightGBM_BAG_L1/T11': 9.997263669967651,
'LightGBM_BAG_L1/T12': 9.382915258407593,
'LightGBM_BAG_L1/T13': 9.197129487991333,
'LightGBM_BAG_L1/T14': 9.055939674377441,
'LightGBM_BAG_L1/T15': 10.260672330856323,
'LightGBM_BAG_L1/T16': 10.561400413513184,
'LightGBM_BAG_L1/T17': 9.649884939193726,
'LightGBM_BAG_L1/T18': 9.483726024627686,
'LightGBM_BAG_L1/T19': 9.555108308792114,
'LightGBM_BAG_L1/T20': 10.566669464111328,
'LightGBM_BAG_L1/T21': 10.074440479278564,
'LightGBM_BAG_L1/T22': 9.555924415588379,
'LightGBM_BAG_L1/T23': 10.137166500091553,
'LightGBM_BAG_L1/T24': 9.292906522750854,
'LightGBM_BAG_L1/T25': 9.386972427368164,
'LightGBM_BAG_L1/T26': 10.196181774139404,
'LightGBM_BAG_L1/T27': 10.115333080291748,
'LightGBM_BAG_L1/T28': 9.465526819229126,
'LightGBM_BAG_L1/T29': 10.453752517700195,
'LightGBM_BAG_L1/T30': 9.833702564239502,
'LightGBM_BAG_L1/T31': 10.546894073486328,
'LightGBM_BAG_L1/T32': 9.766402959823608,
'LightGBM_BAG_L1/T33': 9.88994836807251,
'LightGBM_BAG_L1/T34': 10.786389589309692,
'LightGBM_BAG_L1/T35': 9.5946524143219,
'LightGBM_BAG_L1/T36': 9.486711978912354,
'LightGBM_BAG_L1/T37': 9.946213245391846,
'LightGBM_BAG_L1/T38': 0.21577024459838867,
'LightGBM_BAG_L1/T39': 0.22226357460021973,
'LightGBM_BAG_L1/T40': 0.24070167541503906,
'LightGBM_BAG_L1/T41': 0.2161695957183838,
'LightGBM_BAG_L1/T42': 0.23299813270568848,
'LightGBM_BAG_L1/T43': 0.2020115852355957,
'LightGBM_BAG_L1/T44': 0.2330324649810791,
'LightGBM_BAG_L1/T45': 0.21660733222961426,
'LightGBM_BAG_L1/T46': 0.23194169998168945,
'LightGBM_BAG_L1/T47': 0.22517013549804688,
'LightGBM_BAG_L1/T48': 0.21594905853271484,
'LightGBM_BAG_L1/T49': 0.22269487380981445,
'LightGBM_BAG_L1/T50': 0.22066521644592285,
'LightGBM_BAG_L1/T51': 0.20440292358398438,
'LightGBM_BAG_L1/T52': 0.22507524490356445,
'LightGBM_BAG_L1/T53': 0.21802711486816406,
'LightGBM_BAG_L1/T54': 0.2181713581085205,
'LightGBM_BAG_L1/T55': 0.20281314849853516,
'LightGBM_BAG_L1/T56': 0.2311084270477295,
'LightGBM_BAG_L1/T57': 0.21780705451965332,
'LightGBM_BAG_L1/T58': 0.222639799118042,
'LightGBM_BAG_L1/T59': 0.2253251075744629,
'LightGBM_BAG_L1/T60': 0.2102985382080078,
'LightGBM_BAG_L1/T61': 0.23256182670593262,
'LightGBM_BAG_L1/T62': 0.23053717613220215,
'LightGBM_BAG_L1/T63': 0.23365521430969238,
'LightGBM_BAG_L1/T64': 0.24268293380737305,
'LightGBM_BAG_L1/T65': 0.487515926361084,
'LightGBM_BAG_L1/T66': 0.4128139019012451,
'LightGBM_BAG_L1/T67': 0.24496889114379883,
'LightGBM_BAG_L1/T68': 0.3071892261505127,
'LightGBM_BAG_L1/T69': 0.26998162269592285,
'LightGBM_BAG_L1/T70': 0.25194287300109863,
'LightGBM_BAG_L1/T71': 0.235917329788208,
'LightGBM_BAG_L1/T72': 0.21227812767028809,
'LightGBM_BAG_L1/T73': 0.21178460121154785,
'LightGBM_BAG_L1/T74': 0.22579431533813477,
'LightGBM_BAG_L1/T75': 0.23928332328796387,
'LightGBM_BAG_L1/T76': 0.22036433219909668,
'LightGBM_BAG_L1/T77': 0.21992969512939453,
'LightGBM_BAG_L1/T78': 0.22661113739013672,
'LightGBM_BAG_L1/T79': 0.23014044761657715,
'LightGBM_BAG_L1/T80': 0.2697489261627197,
'LightGBM_BAG_L1/T81': 0.30761265754699707,
'LightGBM_BAG_L1/T82': 0.2956404685974121,
'LightGBM_BAG_L1/T83': 0.20705318450927734,
'LightGBM_BAG_L1/T84': 0.23265504837036133,
'LightGBM_BAG_L1/T85': 0.23516058921813965,
'LightGBM_BAG_L1/T86': 0.23321747779846191,
'LightGBM_BAG_L1/T87': 0.22266268730163574,
'LightGBM_BAG_L1/T88': 0.22332215309143066,
'LightGBM_BAG_L1/T89': 0.22403383255004883,
'LightGBM_BAG_L1/T90': 0.20606493949890137,
'LightGBM_BAG_L1/T91': 0.2107548713684082,
'LightGBM_BAG_L1/T92': 0.24056529998779297,
'LightGBM_BAG_L1/T93': 0.22280097007751465,
'LightGBM_BAG_L1/T94': 0.22036337852478027,
'LightGBM_BAG_L1/T95': 0.2251594066619873,
'LightGBM_BAG_L1/T96': 0.20960021018981934,
'LightGBM_BAG_L1/T97': 0.20604920387268066,
'LightGBM_BAG_L1/T98': 0.20150423049926758,
'LightGBM_BAG_L1/T99': 0.22423100471496582,
'LightGBM_BAG_L1/T100': 0.22870683670043945,
'LightGBM_BAG_L1/T101': 0.22327589988708496,
'LightGBM_BAG_L1/T102': 0.22685742378234863,
'LightGBM_BAG_L1/T103': 0.2358546257019043,
'LightGBM_BAG_L1/T104': 0.2104032039642334,
'LightGBM_BAG_L1/T105': 0.21479249000549316,
'LightGBM_BAG_L1/T106': 0.2222898006439209,
'LightGBM_BAG_L1/T107': 0.2290949821472168,
'LightGBM_BAG_L1/T108': 0.21549582481384277,
'LightGBM_BAG_L1/T109': 0.22116446495056152,
'LightGBM_BAG_L1/T110': 0.20212554931640625,
'LightGBM_BAG_L1/T111': 0.23015689849853516,
'LightGBM_BAG_L1/T112': 0.21552538871765137,
'LightGBM_BAG_L1/T113': 0.23051166534423828,
'LightGBM_BAG_L1/T114': 0.22807598114013672,
'LightGBM_BAG_L1/T115': 0.18915772438049316,
'NeuralNetTorch_BAG_L1/T1': 0.941309928894043,
'NeuralNetTorch_BAG_L1/T2': 1.6394541263580322,
'NeuralNetTorch_BAG_L1/T3': 1.3688507080078125,
'NeuralNetTorch_BAG_L1/T4': 1.2224206924438477,
'NeuralNetTorch_BAG_L1/T5': 2.3058929443359375,
'NeuralNetTorch_BAG_L1/T6': 1.5041708946228027,
'NeuralNetTorch_BAG_L1/T7': 1.118419885635376,
'NeuralNetTorch_BAG_L1/T8': 0.9925944805145264,
'NeuralNetTorch_BAG_L1/T9': 1.1996958255767822,
'NeuralNetTorch_BAG_L1/T10': 1.223811149597168,
'NeuralNetTorch_BAG_L1/T11': 1.879333734512329,
'NeuralNetTorch_BAG_L1/T12': 0.8602015972137451,
'NeuralNetTorch_BAG_L1/T13': 3.418015718460083,
'NeuralNetTorch_BAG_L1/T14': 0.9969353675842285,
'NeuralNetTorch_BAG_L1/T15': 2.2159149646759033,
'NeuralNetTorch_BAG_L1/T16': 2.8555946350097656,
'NeuralNetTorch_BAG_L1/T17': 1.1460907459259033,
'NeuralNetTorch_BAG_L1/T18': 2.7566206455230713,
'WeightedEnsemble_L2': 0.4757227897644043,
'LightGBM_BAG_L2/T1': 11.517523050308228,
'LightGBM_BAG_L2/T2': 10.674018144607544,
'LightGBM_BAG_L2/T3': 10.848548412322998,
'LightGBM_BAG_L2/T4': 10.944711923599243,
'LightGBM_BAG_L2/T5': 10.919796705245972,
'LightGBM_BAG_L2/T6': 10.881124496459961,
'LightGBM_BAG_L2/T7': 10.528796434402466,
'LightGBM_BAG_L2/T8': 10.597357511520386,
'LightGBM_BAG_L2/T9': 10.956775426864624,
'LightGBM_BAG_L2/T10': 11.38678765296936,
'LightGBM_BAG_L2/T11': 10.840951681137085,
'LightGBM_BAG_L2/T12': 10.329079627990723,
'LightGBM_BAG_L2/T13': 10.763923406600952,
'LightGBM_BAG_L2/T14': 10.407933950424194,
'LightGBM_BAG_L2/T15': 10.214085102081299,
'LightGBM_BAG_L2/T16': 11.171836376190186,
'LightGBM_BAG_L2/T17': 11.736985445022583,
'LightGBM_BAG_L2/T18': 10.413777828216553,
'LightGBM_BAG_L2/T19': 11.450664281845093,
'LightGBM_BAG_L2/T20': 10.20365047454834,
'LightGBM_BAG_L2/T21': 10.500128507614136,
'LightGBM_BAG_L2/T22': 10.572536706924438,
'LightGBM_BAG_L2/T23': 10.374094486236572,
'LightGBM_BAG_L2/T24': 10.308635711669922,
'LightGBM_BAG_L2/T25': 11.152453899383545,
'LightGBM_BAG_L2/T26': 11.170250415802002,
'LightGBM_BAG_L2/T27': 10.467259645462036,
'LightGBM_BAG_L2/T28': 11.032371282577515,
'LightGBM_BAG_L2/T29': 0.4594917297363281,
'LightGBM_BAG_L2/T30': 0.3959805965423584,
'LightGBM_BAG_L2/T31': 0.4463169574737549,
'LightGBM_BAG_L2/T32': 0.4218437671661377,
'LightGBM_BAG_L2/T33': 0.37578582763671875,
'LightGBM_BAG_L2/T34': 0.4753706455230713,
'LightGBM_BAG_L2/T35': 0.503826379776001,
'LightGBM_BAG_L2/T36': 0.3657855987548828,
'LightGBM_BAG_L2/T37': 0.5009479522705078,
'LightGBM_BAG_L2/T38': 0.4122910499572754,
'LightGBM_BAG_L2/T39': 0.478163480758667,
'LightGBM_BAG_L2/T40': 0.47332215309143066,
'LightGBM_BAG_L2/T41': 0.3971693515777588,
'LightGBM_BAG_L2/T42': 0.48556995391845703,
'LightGBM_BAG_L2/T43': 0.3502504825592041,
'LightGBM_BAG_L2/T44': 0.48833751678466797,
'LightGBM_BAG_L2/T45': 0.4118661880493164,
'LightGBM_BAG_L2/T46': 0.4566938877105713,
'LightGBM_BAG_L2/T47': 0.43578362464904785,
'LightGBM_BAG_L2/T48': 0.35762476921081543,
'LightGBM_BAG_L2/T49': 0.4408249855041504,
'LightGBM_BAG_L2/T50': 0.43892383575439453,
'LightGBM_BAG_L2/T51': 0.3358285427093506,
'LightGBM_BAG_L2/T52': 0.4217817783355713,
'NeuralNetTorch_BAG_L2/T1': 1.2834010124206543,
'NeuralNetTorch_BAG_L2/T2': 1.731952428817749,
'NeuralNetTorch_BAG_L2/T3': 1.9282355308532715,
'NeuralNetTorch_BAG_L2/T4': 1.477004051208496,
'NeuralNetTorch_BAG_L2/T5': 2.4433677196502686,
'NeuralNetTorch_BAG_L2/T6': 1.7685227394104004,
'NeuralNetTorch_BAG_L2/T7': 1.2231862545013428,
'NeuralNetTorch_BAG_L2/T8': 1.293245553970337,
'NeuralNetTorch_BAG_L2/T9': 1.46917724609375,
'NeuralNetTorch_BAG_L2/T10': 1.6383426189422607,
'NeuralNetTorch_BAG_L2/T11': 2.257392644882202,
'NeuralNetTorch_BAG_L2/T12': 0.9296023845672607,
'NeuralNetTorch_BAG_L2/T13': 3.5259954929351807,
'WeightedEnsemble_L3': 0.3479495048522949,
'LightGBM_BAG_L3/T1': 10.421271562576294,
'LightGBM_BAG_L3/T2': 10.745707035064697,
'LightGBM_BAG_L3/T3': 10.532835960388184,
'LightGBM_BAG_L3/T4': 10.472123861312866,
'LightGBM_BAG_L3/T5': 10.819388151168823,
'LightGBM_BAG_L3/T6': 14.983085870742798,
'LightGBM_BAG_L3/T7': 10.5095853805542,
'LightGBM_BAG_L3/T8': 10.47484827041626,
'LightGBM_BAG_L3/T9': 11.093887329101562,
'LightGBM_BAG_L3/T10': 11.26975154876709,
'LightGBM_BAG_L3/T11': 10.664169788360596,
'LightGBM_BAG_L3/T12': 10.277475595474243,
'LightGBM_BAG_L3/T13': 10.346750974655151,
'LightGBM_BAG_L3/T14': 0.3103153705596924,
'LightGBM_BAG_L3/T15': 0.327251672744751,
'LightGBM_BAG_L3/T16': 0.37508320808410645,
'LightGBM_BAG_L3/T17': 0.3987743854522705,
'LightGBM_BAG_L3/T18': 0.3297741413116455,
'LightGBM_BAG_L3/T19': 0.3348805904388428,
'LightGBM_BAG_L3/T20': 0.32520484924316406,
'LightGBM_BAG_L3/T21': 0.36978673934936523,
'LightGBM_BAG_L3/T22': 0.3102102279663086,
'LightGBM_BAG_L3/T23': 0.3131117820739746,
'LightGBM_BAG_L3/T24': 0.32816410064697266,
'LightGBM_BAG_L3/T25': 0.3730144500732422,
'LightGBM_BAG_L3/T26': 0.3900635242462158,
'LightGBM_BAG_L3/T27': 0.36591362953186035,
'LightGBM_BAG_L3/T28': 0.33516955375671387,
'LightGBM_BAG_L3/T29': 0.3599574565887451,
'LightGBM_BAG_L3/T30': 0.3373551368713379,
'NeuralNetTorch_BAG_L3/T1': 1.2312066555023193,
'NeuralNetTorch_BAG_L3/T2': 1.7005503177642822,
'NeuralNetTorch_BAG_L3/T3': 1.581681489944458,
'NeuralNetTorch_BAG_L3/T4': 1.4623653888702393,
'NeuralNetTorch_BAG_L3/T5': 2.429792881011963,
'NeuralNetTorch_BAG_L3/T6': 1.9207446575164795,
'WeightedEnsemble_L4': 0.3485684394836426},
'model_pred_times': {'LightGBM_BAG_L1/T1': 0.04482841491699219,
'LightGBM_BAG_L1/T2': 0.06255245208740234,
'LightGBM_BAG_L1/T3': 0.05582237243652344,
'LightGBM_BAG_L1/T4': 0.05430006980895996,
'LightGBM_BAG_L1/T5': 0.0587308406829834,
'LightGBM_BAG_L1/T6': 0.05639791488647461,
'LightGBM_BAG_L1/T7': 0.05878496170043945,
'LightGBM_BAG_L1/T8': 0.05945444107055664,
'LightGBM_BAG_L1/T9': 0.053844451904296875,
'LightGBM_BAG_L1/T10': 0.08932852745056152,
'LightGBM_BAG_L1/T11': 0.057440996170043945,
'LightGBM_BAG_L1/T12': 0.0636129379272461,
'LightGBM_BAG_L1/T13': 0.058502197265625,
'LightGBM_BAG_L1/T14': 0.05248904228210449,
'LightGBM_BAG_L1/T15': 0.05943155288696289,
'LightGBM_BAG_L1/T16': 0.06434082984924316,
'LightGBM_BAG_L1/T17': 0.06077766418457031,
'LightGBM_BAG_L1/T18': 0.05007529258728027,
'LightGBM_BAG_L1/T19': 0.057547569274902344,
'LightGBM_BAG_L1/T20': 0.0552067756652832,
'LightGBM_BAG_L1/T21': 0.054090261459350586,
'LightGBM_BAG_L1/T22': 0.04945778846740723,
'LightGBM_BAG_L1/T23': 0.050156354904174805,
'LightGBM_BAG_L1/T24': 0.07278990745544434,
'LightGBM_BAG_L1/T25': 0.05554533004760742,
'LightGBM_BAG_L1/T26': 0.05892205238342285,
'LightGBM_BAG_L1/T27': 0.058286428451538086,
'LightGBM_BAG_L1/T28': 0.053971052169799805,
'LightGBM_BAG_L1/T29': 0.055869340896606445,
'LightGBM_BAG_L1/T30': 0.0651853084564209,
'LightGBM_BAG_L1/T31': 0.06513166427612305,
'LightGBM_BAG_L1/T32': 0.06690144538879395,
'LightGBM_BAG_L1/T33': 0.05134010314941406,
'LightGBM_BAG_L1/T34': 0.06319308280944824,
'LightGBM_BAG_L1/T35': 0.05600881576538086,
'LightGBM_BAG_L1/T36': 0.06562376022338867,
'LightGBM_BAG_L1/T37': 0.04752159118652344,
'LightGBM_BAG_L1/T38': 0.006990671157836914,
'LightGBM_BAG_L1/T39': 0.006960391998291016,
'LightGBM_BAG_L1/T40': 0.0068759918212890625,
'LightGBM_BAG_L1/T41': 0.007127046585083008,
'LightGBM_BAG_L1/T42': 0.0070056915283203125,
'LightGBM_BAG_L1/T43': 0.005974769592285156,
'LightGBM_BAG_L1/T44': 0.00800633430480957,
'LightGBM_BAG_L1/T45': 0.0068209171295166016,
'LightGBM_BAG_L1/T46': 0.0072460174560546875,
'LightGBM_BAG_L1/T47': 0.007654666900634766,
'LightGBM_BAG_L1/T48': 0.006855010986328125,
'LightGBM_BAG_L1/T49': 0.008553028106689453,
'LightGBM_BAG_L1/T50': 0.006351470947265625,
'LightGBM_BAG_L1/T51': 0.006451129913330078,
'LightGBM_BAG_L1/T52': 0.007132768630981445,
'LightGBM_BAG_L1/T53': 0.007700920104980469,
'LightGBM_BAG_L1/T54': 0.006682395935058594,
'LightGBM_BAG_L1/T55': 0.008810043334960938,
'LightGBM_BAG_L1/T56': 0.0067577362060546875,
'LightGBM_BAG_L1/T57': 0.006837129592895508,
'LightGBM_BAG_L1/T58': 0.0069887638092041016,
'LightGBM_BAG_L1/T59': 0.0073719024658203125,
'LightGBM_BAG_L1/T60': 0.0072176456451416016,
'LightGBM_BAG_L1/T61': 0.007396221160888672,
'LightGBM_BAG_L1/T62': 0.007167339324951172,
'LightGBM_BAG_L1/T63': 0.006453752517700195,
'LightGBM_BAG_L1/T64': 0.0067577362060546875,
'LightGBM_BAG_L1/T65': 0.03864240646362305,
'LightGBM_BAG_L1/T66': 0.013461828231811523,
'LightGBM_BAG_L1/T67': 0.008053302764892578,
'LightGBM_BAG_L1/T68': 0.008565664291381836,
'LightGBM_BAG_L1/T69': 0.007619619369506836,
'LightGBM_BAG_L1/T70': 0.007033109664916992,
'LightGBM_BAG_L1/T71': 0.008518695831298828,
'LightGBM_BAG_L1/T72': 0.009573221206665039,
'LightGBM_BAG_L1/T73': 0.006136178970336914,
'LightGBM_BAG_L1/T74': 0.007794618606567383,
'LightGBM_BAG_L1/T75': 0.00719451904296875,
'LightGBM_BAG_L1/T76': 0.007575035095214844,
'LightGBM_BAG_L1/T77': 0.006597757339477539,
'LightGBM_BAG_L1/T78': 0.007218837738037109,
'LightGBM_BAG_L1/T79': 0.00942230224609375,
'LightGBM_BAG_L1/T80': 0.017364501953125,
'LightGBM_BAG_L1/T81': 0.01599740982055664,
'LightGBM_BAG_L1/T82': 0.006402254104614258,
'LightGBM_BAG_L1/T83': 0.0060558319091796875,
'LightGBM_BAG_L1/T84': 0.006954193115234375,
'LightGBM_BAG_L1/T85': 0.012075185775756836,
'LightGBM_BAG_L1/T86': 0.006605386734008789,
'LightGBM_BAG_L1/T87': 0.006609201431274414,
'LightGBM_BAG_L1/T88': 0.006662845611572266,
'LightGBM_BAG_L1/T89': 0.007159233093261719,
'LightGBM_BAG_L1/T90': 0.0067920684814453125,
'LightGBM_BAG_L1/T91': 0.006179094314575195,
'LightGBM_BAG_L1/T92': 0.007559776306152344,
'LightGBM_BAG_L1/T93': 0.006592273712158203,
'LightGBM_BAG_L1/T94': 0.006869792938232422,
'LightGBM_BAG_L1/T95': 0.007148027420043945,
'LightGBM_BAG_L1/T96': 0.006707668304443359,
'LightGBM_BAG_L1/T97': 0.006242513656616211,
'LightGBM_BAG_L1/T98': 0.007392168045043945,
'LightGBM_BAG_L1/T99': 0.00706934928894043,
'LightGBM_BAG_L1/T100': 0.007288217544555664,
'LightGBM_BAG_L1/T101': 0.007936954498291016,
'LightGBM_BAG_L1/T102': 0.006632804870605469,
'LightGBM_BAG_L1/T103': 0.006613969802856445,
'LightGBM_BAG_L1/T104': 0.005826234817504883,
'LightGBM_BAG_L1/T105': 0.00888204574584961,
'LightGBM_BAG_L1/T106': 0.008012771606445312,
'LightGBM_BAG_L1/T107': 0.007217884063720703,
'LightGBM_BAG_L1/T108': 0.006499528884887695,
'LightGBM_BAG_L1/T109': 0.0070536136627197266,
'LightGBM_BAG_L1/T110': 0.006432056427001953,
'LightGBM_BAG_L1/T111': 0.007245540618896484,
'LightGBM_BAG_L1/T112': 0.007080554962158203,
'LightGBM_BAG_L1/T113': 0.00712132453918457,
'LightGBM_BAG_L1/T114': 0.01260232925415039,
'LightGBM_BAG_L1/T115': 0.006214618682861328,
'NeuralNetTorch_BAG_L1/T1': 0.031775712966918945,
'NeuralNetTorch_BAG_L1/T2': 0.03270244598388672,
'NeuralNetTorch_BAG_L1/T3': 0.03454399108886719,
'NeuralNetTorch_BAG_L1/T4': 0.030295372009277344,
'NeuralNetTorch_BAG_L1/T5': 0.03597140312194824,
'NeuralNetTorch_BAG_L1/T6': 0.03450298309326172,
'NeuralNetTorch_BAG_L1/T7': 0.025508403778076172,
'NeuralNetTorch_BAG_L1/T8': 0.026576995849609375,
'NeuralNetTorch_BAG_L1/T9': 0.035176992416381836,
'NeuralNetTorch_BAG_L1/T10': 0.037171125411987305,
'NeuralNetTorch_BAG_L1/T11': 0.041609764099121094,
'NeuralNetTorch_BAG_L1/T12': 0.02891850471496582,
'NeuralNetTorch_BAG_L1/T13': 0.05699324607849121,
'NeuralNetTorch_BAG_L1/T14': 0.025221824645996094,
'NeuralNetTorch_BAG_L1/T15': 0.04282379150390625,
'NeuralNetTorch_BAG_L1/T16': 0.04796481132507324,
'NeuralNetTorch_BAG_L1/T17': 0.03795623779296875,
'NeuralNetTorch_BAG_L1/T18': 0.045452117919921875,
'WeightedEnsemble_L2': 0.0011477470397949219,
'LightGBM_BAG_L2/T1': 0.05970311164855957,
'LightGBM_BAG_L2/T2': 0.07781696319580078,
'LightGBM_BAG_L2/T3': 0.06888103485107422,
'LightGBM_BAG_L2/T4': 0.05910181999206543,
'LightGBM_BAG_L2/T5': 0.06439924240112305,
'LightGBM_BAG_L2/T6': 0.06627464294433594,
'LightGBM_BAG_L2/T7': 0.06974911689758301,
'LightGBM_BAG_L2/T8': 0.06839776039123535,
'LightGBM_BAG_L2/T9': 0.06385278701782227,
'LightGBM_BAG_L2/T10': 0.06974506378173828,
'LightGBM_BAG_L2/T11': 0.07095861434936523,
'LightGBM_BAG_L2/T12': 0.09368443489074707,
'LightGBM_BAG_L2/T13': 0.07300138473510742,
'LightGBM_BAG_L2/T14': 0.10724234580993652,
'LightGBM_BAG_L2/T15': 0.05807232856750488,
'LightGBM_BAG_L2/T16': 0.07395267486572266,
'LightGBM_BAG_L2/T17': 0.0634455680847168,
'LightGBM_BAG_L2/T18': 0.053638458251953125,
'LightGBM_BAG_L2/T19': 0.06345129013061523,
'LightGBM_BAG_L2/T20': 0.06558966636657715,
'LightGBM_BAG_L2/T21': 0.061888694763183594,
'LightGBM_BAG_L2/T22': 0.0582273006439209,
'LightGBM_BAG_L2/T23': 0.05881214141845703,
'LightGBM_BAG_L2/T24': 0.07249259948730469,
'LightGBM_BAG_L2/T25': 0.05895256996154785,
'LightGBM_BAG_L2/T26': 0.07042622566223145,
'LightGBM_BAG_L2/T27': 0.0771629810333252,
'LightGBM_BAG_L2/T28': 0.06526303291320801,
'LightGBM_BAG_L2/T29': 0.007224559783935547,
'LightGBM_BAG_L2/T30': 0.007968425750732422,
'LightGBM_BAG_L2/T31': 0.007632732391357422,
'LightGBM_BAG_L2/T32': 0.007171630859375,
'LightGBM_BAG_L2/T33': 0.011116981506347656,
'LightGBM_BAG_L2/T34': 0.008387565612792969,
'LightGBM_BAG_L2/T35': 0.007893562316894531,
'LightGBM_BAG_L2/T36': 0.011198282241821289,
'LightGBM_BAG_L2/T37': 0.009037017822265625,
'LightGBM_BAG_L2/T38': 0.006924867630004883,
'LightGBM_BAG_L2/T39': 0.00865626335144043,
'LightGBM_BAG_L2/T40': 0.007783412933349609,
'LightGBM_BAG_L2/T41': 0.007379770278930664,
'LightGBM_BAG_L2/T42': 0.007703304290771484,
'LightGBM_BAG_L2/T43': 0.007624626159667969,
'LightGBM_BAG_L2/T44': 0.007793903350830078,
'LightGBM_BAG_L2/T45': 0.008583784103393555,
'LightGBM_BAG_L2/T46': 0.009446382522583008,
'LightGBM_BAG_L2/T47': 0.00810551643371582,
'LightGBM_BAG_L2/T48': 0.008466482162475586,
'LightGBM_BAG_L2/T49': 0.007613420486450195,
'LightGBM_BAG_L2/T50': 0.007368803024291992,
'LightGBM_BAG_L2/T51': 0.007039070129394531,
'LightGBM_BAG_L2/T52': 0.007814884185791016,
'NeuralNetTorch_BAG_L2/T1': 0.07587409019470215,
'NeuralNetTorch_BAG_L2/T2': 0.03550553321838379,
'NeuralNetTorch_BAG_L2/T3': 0.07411432266235352,
'NeuralNetTorch_BAG_L2/T4': 0.06741023063659668,
'NeuralNetTorch_BAG_L2/T5': 0.2763195037841797,
'NeuralNetTorch_BAG_L2/T6': 0.0688178539276123,
'NeuralNetTorch_BAG_L2/T7': 0.07483291625976562,
'NeuralNetTorch_BAG_L2/T8': 0.07549834251403809,
'NeuralNetTorch_BAG_L2/T9': 0.08083772659301758,
'NeuralNetTorch_BAG_L2/T10': 0.07576513290405273,
'NeuralNetTorch_BAG_L2/T11': 0.09225821495056152,
'NeuralNetTorch_BAG_L2/T12': 0.026915311813354492,
'NeuralNetTorch_BAG_L2/T13': 0.09824872016906738,
'WeightedEnsemble_L3': 0.0008761882781982422,
'LightGBM_BAG_L3/T1': 0.05529189109802246,
'LightGBM_BAG_L3/T2': 0.06210684776306152,
'LightGBM_BAG_L3/T3': 0.06968259811401367,
'LightGBM_BAG_L3/T4': 0.06047797203063965,
'LightGBM_BAG_L3/T5': 0.0686500072479248,
'LightGBM_BAG_L3/T6': 0.0760955810546875,
'LightGBM_BAG_L3/T7': 0.07187533378601074,
'LightGBM_BAG_L3/T8': 0.05547356605529785,
'LightGBM_BAG_L3/T9': 0.06760144233703613,
'LightGBM_BAG_L3/T10': 0.06429338455200195,
'LightGBM_BAG_L3/T11': 0.061105966567993164,
'LightGBM_BAG_L3/T12': 0.06363034248352051,
'LightGBM_BAG_L3/T13': 0.07178068161010742,
'LightGBM_BAG_L3/T14': 0.006489753723144531,
'LightGBM_BAG_L3/T15': 0.006696939468383789,
'LightGBM_BAG_L3/T16': 0.007131338119506836,
'LightGBM_BAG_L3/T17': 0.006948947906494141,
'LightGBM_BAG_L3/T18': 0.006304502487182617,
'LightGBM_BAG_L3/T19': 0.0063323974609375,
'LightGBM_BAG_L3/T20': 0.009141921997070312,
'LightGBM_BAG_L3/T21': 0.006935834884643555,
'LightGBM_BAG_L3/T22': 0.006928920745849609,
'LightGBM_BAG_L3/T23': 0.006405830383300781,
'LightGBM_BAG_L3/T24': 0.007336854934692383,
'LightGBM_BAG_L3/T25': 0.006997585296630859,
'LightGBM_BAG_L3/T26': 0.0069582462310791016,
'LightGBM_BAG_L3/T27': 0.007478475570678711,
'LightGBM_BAG_L3/T28': 0.00690460205078125,
'LightGBM_BAG_L3/T29': 0.008630990982055664,
'LightGBM_BAG_L3/T30': 0.0074939727783203125,
'NeuralNetTorch_BAG_L3/T1': 0.06948494911193848,
'NeuralNetTorch_BAG_L3/T2': 0.0382237434387207,
'NeuralNetTorch_BAG_L3/T3': 0.07168054580688477,
'NeuralNetTorch_BAG_L3/T4': 0.06579947471618652,
'NeuralNetTorch_BAG_L3/T5': 0.07570505142211914,
'NeuralNetTorch_BAG_L3/T6': 0.0714726448059082,
'WeightedEnsemble_L4': 0.0007472038269042969},
'num_bag_folds': 6,
'max_stack_level': 4,
'model_hyperparams': {'LightGBM_BAG_L1/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T6': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T7': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T8': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T9': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T10': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T11': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T12': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T13': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T14': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T15': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T16': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T17': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T18': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T19': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T20': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T21': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T22': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T23': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T24': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T25': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T26': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T27': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T28': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T29': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T30': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T31': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T32': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T33': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T34': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T35': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T36': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T37': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T38': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T39': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T40': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T41': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T42': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T43': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T44': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T45': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T46': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T47': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T48': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T49': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T50': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T51': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T52': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T53': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T54': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T55': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T56': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T57': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T58': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T59': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T60': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T61': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T62': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T63': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T64': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T65': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T66': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T67': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T68': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T69': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T70': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T71': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T72': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T73': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T74': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T75': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T76': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T77': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T78': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T79': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T80': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T81': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T82': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T83': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T84': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T85': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T86': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T87': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T88': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T89': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T90': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T91': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T92': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T93': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T94': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T95': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T96': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T97': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T98': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T99': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T100': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T101': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T102': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T103': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T104': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T105': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T106': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T107': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T108': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T109': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T110': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T111': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T112': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T113': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T114': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L1/T115': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T6': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T7': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T8': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T9': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T10': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T11': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T12': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T13': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T14': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T15': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T16': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T17': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L1/T18': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L2': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T6': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T7': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T8': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T9': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T10': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T11': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T12': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T13': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T14': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T15': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T16': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T17': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T18': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T19': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T20': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T21': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T22': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T23': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T24': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T25': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T26': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T27': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T28': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T29': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T30': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T31': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T32': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T33': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T34': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T35': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T36': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T37': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T38': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T39': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T40': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T41': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T42': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T43': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T44': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T45': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T46': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T47': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T48': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T49': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T50': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T51': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L2/T52': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T6': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T7': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T8': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T9': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T10': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T11': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T12': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L2/T13': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L3': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T6': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T7': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T8': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T9': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T10': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T11': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T12': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T13': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T14': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T15': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T16': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T17': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T18': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T19': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T20': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T21': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T22': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T23': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T24': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T25': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T26': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T27': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T28': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T29': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'LightGBM_BAG_L3/T30': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L3/T1': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L3/T2': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L3/T3': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L3/T4': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L3/T5': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'NeuralNetTorch_BAG_L3/T6': {'use_orig_features': True,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True},
'WeightedEnsemble_L4': {'use_orig_features': False,
'max_base_models': 25,
'max_base_models_per_type': 5,
'save_bag_folds': True}},
'leaderboard': model score_val pred_time_val fit_time \
0 WeightedEnsemble_L3 -43.460319 2.306577 381.974152
1 LightGBM_BAG_L2/T24 -43.570961 2.235956 370.239415
2 LightGBM_BAG_L2/T10 -43.937039 2.233209 371.317567
3 LightGBM_BAG_L3/T10 -44.345180 4.141941 673.566589
4 WeightedEnsemble_L4 -44.345180 4.142688 673.915158
.. ... ... ... ...
232 LightGBM_BAG_L1/T95 -166.662518 0.007148 0.225159
233 LightGBM_BAG_L1/T52 -168.978088 0.007133 0.225075
234 LightGBM_BAG_L1/T98 -169.867051 0.007392 0.201504
235 LightGBM_BAG_L1/T97 -171.494836 0.006243 0.206049
236 LightGBM_BAG_L1/T115 -180.546931 0.006215 0.189158
pred_time_val_marginal fit_time_marginal stack_level can_infer \
0 0.000876 0.347950 3 True
1 0.072493 10.308636 2 True
2 0.069745 11.386788 2 True
3 0.064293 11.269752 3 True
4 0.000747 0.348568 4 True
.. ... ... ... ...
232 0.007148 0.225159 1 True
233 0.007133 0.225075 1 True
234 0.007392 0.201504 1 True
235 0.006243 0.206049 1 True
236 0.006215 0.189158 1 True
fit_order
0 200
1 158
2 144
3 210
4 237
.. ...
232 95
233 52
234 98
235 97
236 115
[237 rows x 9 columns]}
Create predictions for train dataset
In [81]:
#predictor_new_hpo_v2
predictor_new_hpov2_train = predictor_new_hpo_v2.predict_proba(train)
rmse_new_hpov2 = mean_squared_error(train['count'], predictor_new_hpov2_train, squared=False)
print('The RMSE for predictor new hpo is: ',rmse_new_hpov2)
The RMSE for predictor new hpo is: 35.513254542697794
Create predictions for test dataset
In [82]:
#predictor_new_hpo_v2
predictions_new_hpo_v2 = predictor_new_hpo_v2.predict_proba(test)
In [83]:
submission_new_hpo_v2 = pd.DataFrame({'datetime': submission['datetime'],
'count': predictions_new_hpo_v2})
submission_new_hpo_v2.loc[submission_new_hpo_v2['count'] < 0, 'count'] = 0
In [84]:
# Same submitting predictions
submission_new_hpo_v2['count'] = submission_new_hpo_v2['count'].fillna(0).astype(int)
submission_new_hpo_v2['count'] = submission_new_hpo_v2['count'].astype(np.int64)
submission_new_hpo_v2.to_csv("submission_new_hpo_v2.csv", index=False)
In [85]:
!kaggle competitions submit -c bike-sharing-demand -f submission_new_hpo_v2.csv -m "new features with hyperparameters v2"
100%|█████████████████████████████████████████| 150k/150k [00:00<00:00, 364kB/s] Successfully submitted to Bike Sharing Demand
In [86]:
!kaggle competitions submissions -c bike-sharing-demand | tail -n +1 | head -n 6
fileName date description status publicScore privateScore --------------------------- ------------------- ------------------------------------ -------- ----------- ------------ submission_new_hpo_v2.csv 2022-05-21 14:01:59 new features with hyperparameters v2 complete 0.54039 0.54039 submission_new_hpo.csv 2022-05-21 12:42:39 new features with hyperparameters complete 0.47659 0.47659 submission_new_features.csv 2022-05-21 12:28:51 new features complete 0.46603 0.46603 submission.csv 2022-05-21 12:08:15 first raw submission complete 1.82081 1.82081
New Score of 0.54039¶
In [89]:
# Taking the top model score from each training run and creating a line plot to show improvement
# You can create these in the notebook and save them to PNG or use some other tool (e.g. google sheets, excel)
fig = pd.DataFrame(
{
"model": ["initial", "add_features", "hpo", "hpo_v2"],
"score": [
predictor.leaderboard(silent=True)['score_val'][0],
predictor_new_features.leaderboard(silent=True)['score_val'][0],
predictor_new_hpo.leaderboard(silent=True)['score_val'][0],
predictor_new_hpo_v2.leaderboard(silent=True)['score_val'][0]],
}
).plot(x="model", y="score", figsize=(8, 6)).get_figure()
fig.savefig('model_train_score.png')
In [90]:
# Take the 3 kaggle scores and creating a line plot to show improvement
fig = pd.DataFrame(
{
"test_eval": ["initial", "add_features", "hpo", "hpo_v2"],
"score": [1.82081, 0.46603, 0.47659, 0.54039]
}
).plot(x="test_eval", y="score", figsize=(8, 6)).get_figure()
fig.savefig('model_test_score.png')
Hyperparameter table¶
In [91]:
# The 3 hyperparameters we tuned with the kaggle score as the result
pd.DataFrame({
"model": ["initial", "add_features", "hpo", "hpo_v2"],
"time": [600, 900, 1200, 1200],
"num_bag_folds": [8, 8, 5, 6],
"num_stack_levels": [3, 3, 2, 2],
"score": [1.82081, 0.46603, 0.47659, 0.54039]
})
Out[91]:
| model | time | num_bag_folds | num_stack_levels | score | |
|---|---|---|---|---|---|
| 0 | initial | 600 | 8 | 3 | 1.82081 |
| 1 | add_features | 900 | 8 | 3 | 0.46603 |
| 2 | hpo | 1200 | 5 | 2 | 0.47659 |
| 3 | hpo_v2 | 1200 | 6 | 2 | 0.54039 |